(I) Introduction to Eigenvectors and Eigenvalues

Although a transformation \(\vec{x} \mapsto A\vec{x}\) may move vectors in a variety of directions, there are special vectors on which the action of \(A\) is quite simple.

Example 1

Find the images of \(\vec{u}\) and \(\vec{v}\) under multiplication by \(A\).

Let \( A = \begin{bmatrix} 3 & -2 \\ 1 & 0 \end{bmatrix} \), \( \vec{u} = \begin{bmatrix} -1 \\ 1 \end{bmatrix} \), and \( \vec{v} = \begin{bmatrix} 2 \\ 1 \end{bmatrix} \).

Solution:

\[ A\vec{v} = \begin{bmatrix} 3 & -2 \\ 1 & 0 \end{bmatrix} \begin{bmatrix} 2 \\ 1 \end{bmatrix} = \begin{bmatrix} 4 \\ 2 \end{bmatrix} = 2\begin{bmatrix} 2 \\ 1 \end{bmatrix} = 2\vec{v} \]

Notice that \(A\vec{v}\) is simply a scalar multiple of \(\vec{v}\). The transformation merely scaled the vector.

\[ A\vec{u} = \begin{bmatrix} 3 & -2 \\ 1 & 0 \end{bmatrix} \begin{bmatrix} -1 \\ 1 \end{bmatrix} = \begin{bmatrix} -5 \\ -1 \end{bmatrix} \]

In contrast, \(\vec{u}\) was moved to a completely different direction.

This section studies equations such as \(A\vec{v} = 2\vec{v}\).

Definition

An eigenvector of an \(n \times n\) matrix \(A\) is a nonzero vector \(\vec{x}\) such that:

\[ A\vec{x} = \lambda\vec{x} \]

for some scalar \(\lambda\). A scalar \(\lambda\) is called an eigenvalue of \(A\) if there is a nontrivial solution \(\vec{x}\) for \(A\vec{x} = \lambda\vec{x}\). Such an \(\vec{x}\) is called an eigenvector corresponding to \(\lambda\).

We can rewrite the definition equation as:

\[ (A - \lambda I)\vec{x} = \vec{0} \quad (*) \]
Example 2

Let \( A = \begin{bmatrix} 1 & 6 \\ 5 & 2 \end{bmatrix} \), \( \vec{u} = \begin{bmatrix} -6 \\ 5 \end{bmatrix} \), and \( \vec{v} = \begin{bmatrix} 3 \\ 2 \end{bmatrix} \). Are \(\vec{u}\) and \(\vec{v}\) eigenvectors of \(A\)?

Solution:

\[ A\vec{u} = \begin{bmatrix} 1 & 6 \\ 5 & 2 \end{bmatrix} \begin{bmatrix} -6 \\ 5 \end{bmatrix} = \begin{bmatrix} 24 \\ -20 \end{bmatrix} = -4 \begin{bmatrix} -6 \\ 5 \end{bmatrix} \]

Yes, \(\vec{u}\) is an eigenvector corresponding to the eigenvalue \(\lambda = -4\).

\[ A\vec{v} = \begin{bmatrix} 1 & 6 \\ 5 & 2 \end{bmatrix} \begin{bmatrix} 3 \\ 2 \end{bmatrix} = \begin{bmatrix} 15 \\ 19 \end{bmatrix} \neq \lambda \begin{bmatrix} 3 \\ 2 \end{bmatrix} \]

No, \(\vec{v}\) is not an eigenvector of \(A\).

(II) Eigenspaces

Example 3

Is \(\lambda = 3\) an eigenvalue of \( A = \begin{bmatrix} 1 & 2 & 2 \\ 3 & -2 & 1 \\ 0 & 1 & 1 \end{bmatrix} \)? If so, find one corresponding eigenvector.

Solution: Check if \((A - 3I)\vec{x} = \vec{0}\) has a nontrivial solution.

\[ A - 3I = \begin{bmatrix} 1-3 & 2 & 2 \\ 3 & -2-3 & 1 \\ 0 & 1 & 1-3 \end{bmatrix} = \begin{bmatrix} -2 & 2 & 2 \\ 3 & -5 & 1 \\ 0 & 1 & -2 \end{bmatrix} \]

Row reduce the matrix:

\( \begin{bmatrix} -2 & 2 & 2 \\ 3 & -5 & 1 \\ 0 & 1 & -2 \end{bmatrix} \sim \begin{bmatrix} 1 & -1 & -1 \\ 0 & 1 & -2 \\ 0 & 0 & 0 \end{bmatrix} \sim \begin{bmatrix} 1 & 0 & -3 \\ 0 & 1 & -2 \\ 0 & 0 & 0 \end{bmatrix} \)

The equations are \(x_1 - 3x_3 = 0\) and \(x_2 - 2x_3 = 0\), with \(x_3\) as a free variable. The general solution is:

\[ \vec{x} = x_3 \begin{bmatrix} 3 \\ 2 \\ 1 \end{bmatrix} \]

Since there is a nontrivial solution, \(\lambda = 3\) is an eigenvalue of \(A\). The vector \(\begin{bmatrix} 3 \\ 2 \\ 1 \end{bmatrix}\) is an eigenvector corresponding to \(\lambda = 3\).

Warning

Although row reduction was used in Example 3 to find eigenvectors, it cannot be used to find eigenvalues. An echelon form of \(A\) usually does not display the eigenvalues of \(A\).

An eigenvalue of an \(n \times n\) matrix \(A\) exists if and only if the equation \((A - \lambda I)\vec{x} = \vec{0}\) has a nontrivial solution. This is true if and only if \(\det(A - \lambda I) = 0\) (which will be discussed in Section 5.2).

Definition

The set of all solutions of \((A - \lambda I)\vec{x} = \vec{0}\) is called the eigenspace of \(A\) corresponding to \(\lambda\). It is the null space of \(A - \lambda I\).

Since it is a null space, it is a subspace of \(\mathbb{R}^n\).

The eigenspace corresponding to \(\lambda\) consists of the zero vector and all the eigenvectors corresponding to \(\lambda\).

Note: In Example 3, the eigenspace corresponding to \(\lambda = 3\) is a line through the origin and the vector \(\begin{bmatrix} 3 \\ 2 \\ 1 \end{bmatrix}\) in \(\mathbb{R}^3\).

Example 4

Let \( A = \begin{bmatrix} 4 & 2 & 3 \\ -1 & 1 & -3 \\ 2 & 4 & 9 \end{bmatrix} \). An eigenvalue is \(3\). Find a basis for the corresponding eigenspace.

Solution: Form \(A - 3I\) and find its null space.

\[ A - 3I = \begin{bmatrix} 1 & 2 & 3 \\ -1 & -2 & -3 \\ 2 & 4 & 6 \end{bmatrix} \sim \begin{bmatrix} 1 & 2 & 3 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix} \]

The equation is \(x_1 + 2x_2 + 3x_3 = 0 \implies x_1 = -2x_2 - 3x_3\). Variables \(x_2\) and \(x_3\) are free.

\[ \vec{x} = x_2 \begin{bmatrix} -2 \\ 1 \\ 0 \end{bmatrix} + x_3 \begin{bmatrix} -3 \\ 0 \\ 1 \end{bmatrix} = x_2\vec{v}_1 + x_3\vec{v}_2 \]

A basis for the eigenspace corresponding to \(3\) is \(\{\vec{v}_1, \vec{v}_2\}\).

This eigenspace is a 2-dimensional subspace of \(\mathbb{R}^3\), and it represents a plane through the origin, \(\vec{v}_1\), and \(\vec{v}_2\).

(III) Theorems and Properties

The following theorem describes one of the few special cases in which eigenvalues can be found easily and precisely. (The general calculation of eigenvalues will be discussed in Section 5.2).

Theorem 1

The eigenvalues of a triangular matrix are the entries on its main diagonal.

Example 5

Let \( A = \begin{bmatrix} 3 & 6 & -8 \\ 0 & 0 & 6 \\ 0 & 0 & 2 \end{bmatrix} \) and \( B = \begin{bmatrix} 0 & 0 & 0 \\ -2 & 1 & 0 \\ 5 & 3 & 4 \end{bmatrix} \).

  • The eigenvalues of \(A\) are: \(3, 0, 2\).
  • The eigenvalues of \(B\) are: \(0, 1, 4\).

Notice that eigenvalues can be \(0\).

What does it mean for \(A\) to have an eigenvalue of \(0\)?
It means \(A\vec{x} = 0\vec{x} \implies A\vec{x} = \vec{0}\) has a nontrivial solution. If \(A\vec{x} = \vec{0}\) has a nontrivial solution, then \(A\) is not invertible.

Conclusion: \(0\) is an eigenvalue of \(A\) if and only if \(A\) is not invertible.

Theorem 2

If \(\vec{v}_1, \dots, \vec{v}_p\) are eigenvectors that correspond to distinct eigenvalues \(\lambda_1, \dots, \lambda_p\) of an \(n \times n\) matrix \(A\), then the set \(\{\vec{v}_1, \dots, \vec{v}_p\}\) is linearly independent.

Example 6

(1) Let \(\lambda\) be an eigenvalue of an \(n \times n\) matrix \(A\). Find an eigenvalue of \(2A\) and \(A^3\).

Solution: Since \(\lambda\) is an eigenvalue, \(A\vec{x} = \lambda\vec{x}\) for some \(\vec{x} \neq \vec{0}\).

For \(2A\):

\[ (2A)\vec{x} = 2(A\vec{x}) = 2(\lambda\vec{x}) = (2\lambda)\vec{x} \]

So, \(2\lambda\) is an eigenvalue of \(2A\).

For \(A^3\):

\[ A^2\vec{x} = A(A\vec{x}) = A(\lambda\vec{x}) = \lambda(A\vec{x}) = \lambda(\lambda\vec{x}) = \lambda^2\vec{x} \] \[ A^3\vec{x} = A(A^2\vec{x}) = A(\lambda^2\vec{x}) = \lambda^2(A\vec{x}) = \lambda^2(\lambda\vec{x}) = \lambda^3\vec{x} \]

So, \(\lambda^3\) is an eigenvalue of \(A^3\).

(2) If \(\lambda\) is an eigenvalue of an \(n \times n\) invertible matrix \(A\), then show that \(\lambda^{-1}\) is an eigenvalue of \(A^{-1}\).

Proof:

Let \(\vec{x}\) be an eigenvector corresponding to \(\lambda\). Thus, \(A\vec{x} = \lambda\vec{x}\).

Since \(A\) is invertible, \(\lambda \neq 0\).

Multiply both sides on the left by \(A^{-1}\):

\[ A^{-1}(A\vec{x}) = A^{-1}(\lambda\vec{x}) \] \[ I\vec{x} = \lambda(A^{-1}\vec{x}) \] \[ \vec{x} = \lambda(A^{-1}\vec{x}) \]

Divide by \(\lambda\) (which is valid since \(\lambda \neq 0\)):

\[ A^{-1}\vec{x} = \lambda^{-1}\vec{x} \]

Therefore, \(\lambda^{-1}\) is an eigenvalue of \(A^{-1}\).