7  Eigenvalues, Eigenvectors and all that

“There is hardly any theory which is more elementary than linear algebra, in spite of the fact that generations of professors and textbook writers have obscured its simplicity by preposterous calculations with matrices.”
– Jean Dieudonné

7.1 Eigenvalues and Eigenvectors

Eigenvalues and Eigenvectors

A square n \times n matrix \mathbf{A} is said to have an eigenvalue-eigenvector pair (\lambda, \mathbf{v}) if there is a scalar \lambda \in \mathbb{R} and a vector \mathbf{0} \ne \mathbf{v} \in \mathbb{R}^n such that \mathbf{A} \mathbf{v} = \lambda \mathbf{v}.

You should now notice that a n \times n matrix is singular iff it has 0 as an eigenvalue.

Exercise 7.1  

  1. Find the eigenvalues and eigenvectors of the matrices A=\begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix}, B=\begin{bmatrix} -4 & -4 & 2\\ 3 & 4 & -1 \\ -3 & -2 & 3 \end{bmatrix}
  2. Find a 3 \times 3 matrix \mathbf{A} such that one of the eigenvalues of \mathbf{A} is zero.
Characterizations of eigenvalue

TFAE:

  1. \lambda \in \mathbb{C} is an eigenvalue of $n times n $ matrix \mathbf{A}

  2. (\mathbf{A} - \lambda \mathbf{I}) \mathbf{v}=\mathbf{0} has a non-trivial solution

  3. \mathbf{v} \in \mathcal{N} (\mathbf{A} - \lambda \mathbf{I})

  4. \mathbf{A} - \lambda \mathbf{I} is singular

  5. det (\mathbf{A} - \lambda \mathbf{I}) =0 (characteristic equation)

  6. \textbf{rank} (\mathbf{A} - \lambda \mathbf{I}) < n

  1. Real matrices can have complex eigenvalues.
  2. A n \times n matrix has at least one and at most n distinct eigenvalues
  3. The sum of the eigenvalues of a matrix is equal to its trace.
  4. The product of the eigenvalues of a matrix is equal to its determinant.
Algebraic and Geometric Multiplicity

Let \lambda be an eigenvalue of a n \times n matrix \mathbf{A}. Then:

  1. The algebraic multiplicity of \lambda is its multiplicty as a root of the characteristic equation
  2. The geometric multiplicity of \lambda is the dimension of the eigenspace \mathcal{N} (\mathbf{A} - \lambda \mathbf{I})

It can be shown that, for any eigenvalue \lambda of a n \times n matrix \mathbf{A}, geometric multiplicty \le algebraic multiplicity.

You should now notice that the eigenspace of an eigenvalue with algebraic multiplicty 1 must be 1-dimensional.

Exercise 7.2 Prove that geometric multiplicty is bounded above by algebraic multiplicity.

7.2 Eigenbasis (A Basis consisting of Eigenvectors)

Eigenvector Bases (when all eigenvalues are distinct)
  1. If a n \times n matrix has n distinct \textbf{real} eigenvalues, then the corresponding \textbf{real} eigenvectors form a basis of \mathbb{R}^n
  2. If a n \times n matrix has n distinct \textbf{real or complex} eigenvalues, then the corresponding eigenvectors form a basis of \mathbb{C}^n
Spanning set of Eigenvectors (when the two multiplicities coincide)

Let \mathbf{A} be a n \times n matrix.

TFAE:

  1. For each of the eigenvalues of \mathbf{A}, the algebraic and geometric multiplicites coincide

  2. The eigenvectors span \mathbb{C}^n

7.3 Diagonalization

Similarity and diagonalizability
  1. Matrices \mathbf{A} and \mathbf{B} are said to be similar if there is a non-singular matrix \mathbf{S} such that \mathbf{B= S^{-1}A S}
  2. A matrix \mathbf{A} is diagonalizable iff it is similar to a diagonal matrix

A n \times n matrix \mathbf{A} corresponding to a linear transformation T:\mathbb{R}^n \to \mathbb{R}^n is diagonalizable iff there is a diagonal matrix \mathbb{B} of T with respect to some basis.

Diagonalizability = Existance of Eigenbasis

A matrix is diagonalizable iff it has a basis consisting of eigenvectors

  1. From the last section we see that a n \times n matrix with n distinct eigenvalues is diagonalizable.
  2. We also see from the last section that a n \times n matrix is diagonalizable iff the geometric multiplicities of the eigenvalues add up to n

Exercise 7.3  

  1. Let T:\mathbb{R}^2 \to \mathbb{R}^2 be counter-clockwise rotation by \frac{\pi}{2}. Is the matrix corresponding to T diagonalizable?
  2. Let T:\mathbb{R}^2 \to \mathbb{R}^2 be counter-clockwise rotation by \pi. Is the matrix corresponding to T diagonalizable?

We have figured out exactly when a square matrix is diagonalizable. A related question is : when is a square matrox diagonalizable with an orthonormal eigenbasis. The answer is given by the following theorem.

Spectral Theorem

A (square) matrix is diagonalizable with an orthonormal eigenbasis iff it is symmetric

A symmetric n \times n matrix has n real eigenvalues (counting their algebraic multiplicities)

To orthonormally diagonalize a symmetric matrix, we find the eigenvalues of the matrix and a basis for each eigenspace, and then (using the Gram-Schmidt process) find an orthonormal basis of each eigenspace. Finally, assemble the orthonormal bases found above to form an othonormal eigenbasis.

Exercise 7.4 For the matrix \mathbf{A} below, find an orthogonal matrix \mathbf{S} such that \mathbf{S^{-1}AS} is diagonal A=\begin{bmatrix} 1 & 1 & 1 \\ 1 & 1 & 1 \\ 1& 1 & 1 \end{bmatrix}