Section 2.2 & 2.3 The Inverse of a Matrix

(I) Introduction and Properties of Inverses

Definition

An $n \times n$ matrix $A$ is invertible if there is an $n \times n$ matrix $C$ such that:

AC = I_n \quad \text{and} \quad CA = I_n

A matrix that is not invertible is called a singular matrix. A matrix that is invertible is called a nonsingular matrix.

Theorem 4

Let $ A = \begin{bmatrix} a & b \\ c & d \end{bmatrix} $.

If $ad - bc \neq 0$, then $A$ is invertible and: $A^{-1} = \frac{1}{ad - bc} \begin{bmatrix} d & -b \\ -c & a \end{bmatrix}$
If $ad - bc = 0$, then $A$ is not invertible ($A$ is singular).

The quantity $ad - bc$ is called the determinant of $A$, denoted as $\det A = ad - bc$.

Note: Theorem 4 says that a $2 \times 2$ matrix $A$ is invertible if and only if $\det A \neq 0$. We will generalize this conclusion to any $n \times n$ matrix later.

Theorem 5

If $A$ is an invertible $n \times n$ matrix, then for each $\vec{b}$ in $\mathbb{R}^n$, the equation $A\vec{x} = \vec{b}$ has the unique solution:

\vec{x} = A^{-1}\vec{b}

(Mainly used for algebraic calculation and formula derivations).

Example 1

(1) Find the inverse of $ A = \begin{bmatrix} 3 & -4 \\ 5 & -6 \end{bmatrix} $.

Solution: First, find the determinant:

\[ \det A = (3)(-6) - (-4)(5) = -18 + 20 = 2 \]

Since $\det A \neq 0$, the matrix is invertible:

\[ A^{-1} = \frac{1}{2} \begin{bmatrix} -6 & 4 \\ -5 & 3 \end{bmatrix} = \begin{bmatrix} -3 & 2 \\ -\frac{5}{2} & \frac{3}{2} \end{bmatrix} \]

(2) Use Part (1) and Theorem 5 to solve the systems:

\[ \begin{cases} 3x_1 - 4x_2 = 3 \\ 5x_1 - 6x_2 = 7 \end{cases} \]

\[ \begin{cases} 3y_1 - 4y_2 = 5 \\ 5y_1 - 6y_2 = 9 \end{cases} \]

Solution: For the first system with $\vec{b}_1 = \begin{bmatrix} 3 \\ 7 \end{bmatrix}$:

\[ \vec{x} = A^{-1}\vec{b}_1 = \begin{bmatrix} -3 & 2 \\ -\frac{5}{2} & \frac{3}{2} \end{bmatrix} \begin{bmatrix} 3 \\ 7 \end{bmatrix} = \begin{bmatrix} 5 \\ 3 \end{bmatrix} \]

For the second system with $\vec{b}_2 = \begin{bmatrix} 5 \\ 9 \end{bmatrix}$:

\[ \vec{y} = A^{-1}\vec{b}_2 = \begin{bmatrix} -3 & 2 \\ -\frac{5}{2} & \frac{3}{2} \end{bmatrix} \begin{bmatrix} 5 \\ 9 \end{bmatrix} = \begin{bmatrix} 3 \\ 1 \end{bmatrix} \]

Alternatively: The equations can also be solved simultaneously through an augmented matrix combining both $\vec{b}_1$ and $\vec{b}_2$:

\[ [A \mid \vec{b}_1 \ \vec{b}_2] = \begin{bmatrix} 3 & -4 & 3 & 5 \\ 5 & -6 & 7 & 9 \end{bmatrix} \sim \begin{bmatrix} 1 & -1 & 2 & 2 \\ 0 & -1 & -3 & -1 \end{bmatrix} \sim \begin{bmatrix} 1 & 0 & 5 & 3 \\ 0 & 1 & 3 & 1 \end{bmatrix} \]

Theorem 6 (Properties for Inverses)

If $A$ is an invertible matrix, then $A^{-1}$ is invertible and $(A^{-1})^{-1} = A$.
If $A$ and $B$ are $n \times n$ invertible matrices, then so is $AB$, and: $(AB)^{-1} = B^{-1}A^{-1}$
If $A$ is invertible, then so is $A^T$, and $(A^T)^{-1} = (A^{-1})^T$.

Generalizing Theorem 6 (2), we have:

(A_1 A_2 \dots A_n)^{-1} = A_n^{-1} \dots A_2^{-1} A_1^{-1}

The product of $n$ invertible matrices is invertible, and the inverse is the product of their inverses in the reverse order.

Note: If $A$ is invertible, $AB = AC$ implies $B = C$.

Proof: Multiply both sides on the left by $A^{-1}$:

\[ A^{-1}(AB) = A^{-1}(AC) \implies I_n B = I_n C \implies B = C \]

(II) Elementary Matrices and an Algorithm for Finding $A^{-1}$

An elementary matrix $E$ is one that is obtained by performing a single elementary row operation on an identity matrix. Each elementary matrix is invertible.

For example: Swapping row 1 and row 2 of $I_3$:

\[ I_3 = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \xrightarrow{R_1 \leftrightarrow R_2} E_1 = \begin{bmatrix} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{bmatrix} \]

If we apply this elementary matrix to a general matrix $A = \begin{bmatrix} a & b & c \\ d & e & f \\ g & h & i \end{bmatrix}$:

\[ E_1 A = \begin{bmatrix} d & e & f \\ a & b & c \\ g & h & i \end{bmatrix} \]

Fact: If an elementary row operation is performed on an $m \times n$ matrix $A$, the resulting matrix can be written as $EA$, where the $m \times m$ matrix $E$ is created by performing the exact same row operation on $I_m$.

Theorem 7

An $n \times n$ matrix $A$ is invertible if and only if $A$ is row equivalent to $I_n$, and any sequence of elementary row operations that reduces $A$ to $I_n$ also transforms $I_n$ to $A^{-1}$.

An Algorithm for Finding $A^{-1}$

Row reduce the augmented matrix $[A \mid I_n]$. If $A$ is invertible (i.e., $A$ is row equivalent to $I_n$), the reduction will lead to $[I_n \mid A^{-1}]$.

Example 2

Find the inverse of $ A = \begin{bmatrix} 0 & 1 & 2 \\ 1 & 0 & 3 \\ 4 & -3 & 8 \end{bmatrix} $.

Solution: Form the augmented matrix $[A \mid I_3]$ and row reduce:

\[ [A \mid I_3] = \begin{bmatrix} 0 & 1 & 2 & | & 1 & 0 & 0 \\ 1 & 0 & 3 & | & 0 & 1 & 0 \\ 4 & -3 & 8 & | & 0 & 0 & 1 \end{bmatrix} \sim \begin{bmatrix} 1 & 0 & 3 & | & 0 & 1 & 0 \\ 0 & 1 & 2 & | & 1 & 0 & 0 \\ 0 & -3 & -4 & | & 0 & -4 & 1 \end{bmatrix} \] \[ \sim \begin{bmatrix} 1 & 0 & 3 & | & 0 & 1 & 0 \\ 0 & 1 & 2 & | & 1 & 0 & 0 \\ 0 & 0 & 2 & | & 3 & -4 & 1 \end{bmatrix} \sim \begin{bmatrix} 1 & 0 & 3 & | & 0 & 1 & 0 \\ 0 & 1 & 2 & | & 1 & 0 & 0 \\ 0 & 0 & 1 & | & \frac{3}{2} & -2 & \frac{1}{2} \end{bmatrix} \] \[ \sim \begin{bmatrix} 1 & 0 & 0 & | & -\frac{9}{2} & 7 & -\frac{3}{2} \\ 0 & 1 & 0 & | & -2 & 4 & -1 \\ 0 & 0 & 1 & | & \frac{3}{2} & -2 & \frac{1}{2} \end{bmatrix} \]

Therefore, the inverse is:

\[ A^{-1} = \begin{bmatrix} -\frac{9}{2} & 7 & -\frac{3}{2} \\ -2 & 4 & -1 \\ \frac{3}{2} & -2 & \frac{1}{2} \end{bmatrix} \] Check: $AA^{-1}=I_3$.

Section 2.3: Characterization of Inverse Matrices

Theorem 8 (The Invertible Matrix Theorem)

Let $A$ be a square $n \times n$ matrix. Then the following statements are equivalent (i.e., for a given matrix $A$, the statements are either all true or all false):

$A$ is an invertible matrix.
$A$ is row equivalent to the $n \times n$ identity matrix $I_n$.
$A$ has $n$ pivot positions. (This implies $A$ has a pivot in every row and a pivot in every column).
The equation $A\vec{x} = \vec{0}$ has only the trivial solution.
The columns of $A$ form a linearly independent set.
The linear transformation $\vec{x} \mapsto A\vec{x}$ is one-to-one.
The equation $A\vec{x} = \vec{b}$ has at least one solution for each $\vec{b}$ in $\mathbb{R}^n$.
The columns of $A$ span $\mathbb{R}^n$.
The linear transformation $\vec{x} \mapsto A\vec{x}$ maps $\mathbb{R}^n$ onto $\mathbb{R}^n$.
There is an $n \times n$ matrix $C$ such that $CA = I_n$.
There is an $n \times n$ matrix $D$ such that $AD = I_n$.
$A^T$ is an invertible matrix.

Note: Let $A$ and $B$ be square matrices. If $AB = I$, then $A$ and $B$ are both invertible with $B = A^{-1}$ and $A = B^{-1}$.

The Invertible Matrix Theorem (IMT) applies ONLY to square matrices.

Example 3

Determine if the following matrices are invertible.

(1) $ \begin{bmatrix} 1 & 0 & -2 \\ 3 & 0 & -2 \\ -5 & 0 & 9 \end{bmatrix} $

Answer: No. Since the second column is all zeros, the columns do not form a linearly independent set.

(2) $ \begin{bmatrix} 0 & 0 & -2 \\ 3 & 0 & -2 \\ -5 & -1 & 0 \end{bmatrix} $

Answer: Row reducing the matrix yields:

\[ \begin{bmatrix} 0 & 0 & -2 \\ 3 & 0 & -2 \\ -5 & -1 & 0 \end{bmatrix} \sim \begin{bmatrix} 1 & 0 & -2 \\ 0 & 1 & 4 \\ 0 & -1 & -1 \end{bmatrix} \sim \begin{bmatrix} 1 & 0 & -2 \\ 0 & 1 & 4 \\ 0 & 0 & 3 \end{bmatrix} \]

$A$ has 3 pivot positions, and hence is invertible by the Invertible Matrix Theorem.

Invertible Linear Transformations

A linear transformation $T: \mathbb{R}^n \rightarrow \mathbb{R}^n$ is said to be invertible if there exists a function $S: \mathbb{R}^n \rightarrow \mathbb{R}^n$ such that:

S(T(\vec{x})) = \vec{x} \quad \text{for all } \vec{x} \text{ in } \mathbb{R}^n \] \[ T(S(\vec{x})) = \vec{x} \quad \text{for all } \vec{x} \text{ in } \mathbb{R}^n

The function $S$ is called the inverse of $T$ ($S = T^{-1}$).

Conceptually: $A$ transforms $\vec{x} \mapsto A\vec{x}$. Multipling by $A^{-1}$ transforms $A\vec{x}$ back to $\vec{x}$.

Theorem 9

Let $T: \mathbb{R}^n \rightarrow \mathbb{R}^n$ be a linear transformation and let $A$ be the standard matrix for $T$. Then $T$ is invertible if and only if $A$ is invertible.

In that case, the linear transformation $S$ given by $S(\vec{x}) = A^{-1}\vec{x}$ is the unique function satisfying the invertible conditions above.

Example 4

If the columns of a $7 \times 7$ matrix $D$ are linearly independent, what can you say about the solutions of $D\vec{x} = \vec{b}$? Why?

Solution:

By the Invertible Matrix Theorem, since the columns of the square matrix $D$ are linearly independent, $D$ is invertible.

Thus, $D\vec{x} = \vec{b}$ has a solution for each $\vec{b}$ in $\mathbb{R}^7$.

Even better, $D\vec{x} = \vec{b}$ has a unique solution for each $\vec{b}$ in $\mathbb{R}^7$ by Theorem 5 in Section 2.2 ($\vec{x} = D^{-1}\vec{b}$).

Section 2.2 & 2.3