(I) Definition
Subspace of \(\mathbb{R}^n\)
A subspace of \(\mathbb{R}^n\) is any set \(H\) in \(\mathbb{R}^n\) that has the following 3 properties:
- The zero vector \(\vec{0}\) is in \(H\).
- For each \(\vec{u}\) and \(\vec{v}\) in \(H\), the sum \(\vec{u} + \vec{v}\) is in \(H\).
- For each \(\vec{u}\) in \(H\) and each scalar \(c\), the scalar multiple \(c\vec{u}\) is in \(H\).
In words, a subspace is closed under vector addition and scalar multiplication.
If \(\vec{v}_1\) and \(\vec{v}_2\) are in \(\mathbb{R}^n\) and \(H = \text{Span}\{\vec{v}_1, \vec{v}_2\}\), show that \(H\) is a subspace of \(\mathbb{R}^n\).
Proof:
- \(\vec{0} = 0\vec{v}_1 + 0\vec{v}_2 \in H\).
- Take two arbitrary vectors \(\vec{u}\) and \(\vec{v}\) in \(H\).
\(\vec{u} = c_1\vec{v}_1 + c_2\vec{v}_2\)
\(\vec{v} = c_3\vec{v}_1 + c_4\vec{v}_2\)
\(\vec{u} + \vec{v} = (c_1 + c_3)\vec{v}_1 + (c_2 + c_4)\vec{v}_2 \in H\). - For any scalar \(c\),
\(c\vec{u} = c(c_1\vec{v}_1 + c_2\vec{v}_2) = (cc_1)\vec{v}_1 + (cc_2)\vec{v}_2 \in H\).
Therefore, \(H\) is a subspace of \(\mathbb{R}^n\).
If we generalize Example 1, then we have:
For \(\vec{v}_1, \dots, \vec{v}_p\) in \(\mathbb{R}^n\), \(\text{Span}\{\vec{v}_1, \dots, \vec{v}_p\}\) is a subspace of \(\mathbb{R}^n\).
Notes:
- The set containing only the zero vector is called the zero subspace.
- There are two common ways of proving a subset \(H \subseteq \mathbb{R}^n\) is a subspace of \(\mathbb{R}^n\):
- Use the definition directly.
- Show that the set can be written as \(\text{Span}\{\vec{v}_1, \dots, \vec{v}_p\}\).
(II) Column Space and Null Space of a Matrix
Column Space (Col \(A\))
The column space of a matrix \(A\), denoted \(\text{Col } A\), is the set of all linear combinations of the columns of \(A\).
Facts:
- \(\text{Col } A = \text{Span}\{\vec{a}_1, \dots, \vec{a}_n\}\) if \(A = \begin{bmatrix} \vec{a}_1 & \dots & \vec{a}_n \end{bmatrix}\) is an \(m \times n\) matrix.
- \(\text{Col } A\) is a subspace of \(\mathbb{R}^m\).
- \(\text{Col } A = \mathbb{R}^m\) if the columns of \(A\) span \(\mathbb{R}^m\); otherwise, \(\text{Col } A\) is only a part of \(\mathbb{R}^m\).
- \(\text{Col } A\) is the set of all \(\vec{b}\) for which \(A\vec{x} = \vec{b}\) has a solution. (\(\vec{b}\) is in \(\text{Col } A \iff A\vec{x} = \vec{b}\) has a solution).
Null Space (Nul \(A\))
The null space of \(A\), denoted \(\text{Nul } A\), is the set of all solutions to the homogeneous equation \(A\vec{x} = \vec{0}\).
Theorem 12
The null space of an \(m \times n\) matrix \(A\) is a subspace of \(\mathbb{R}^n\).
(It can be proved using the definition of subspace. Try on your own.)
To determine if \(\vec{u}\) is in \(\text{Nul } A\), simply check if \(A\vec{u} = \vec{0}\).
Let \( A = \begin{bmatrix} 1 & -3 & -4 \\ -4 & 6 & -2 \\ -3 & 7 & 6 \end{bmatrix} \), \(\vec{b} = \begin{bmatrix} 5 \\ 3 \\ -1 \end{bmatrix}\), and \(\vec{u} = \begin{bmatrix} 5 \\ 3 \\ -1 \end{bmatrix}\).
(1) Determine whether \(\vec{b}\) is in \(\text{Col } A\).
Solution: Construct the augmented matrix and reduce it to Echelon Form (REF):
\[ \begin{bmatrix} 1 & -3 & -4 & | & 5 \\ -4 & 6 & -2 & | & 3 \\ -3 & 7 & 6 & | & -1 \end{bmatrix} \rightarrow \begin{bmatrix} 1 & -3 & -4 & | & 5 \\ 0 & -6 & -18 & | & 15 \\ 0 & 0 & 0 & | & 0 \end{bmatrix} \]The system is consistent. Therefore, \(\vec{b}\) is in \(\text{Col } A\).
(2) Determine whether \(\vec{u}\) is in \(\text{Nul } A\).
Solution: Compute the product \(A\vec{u}\):
\[ A\vec{u} = \begin{bmatrix} 1 & -3 & -4 \\ -4 & 6 & -2 \\ -3 & 7 & 6 \end{bmatrix} \begin{bmatrix} 5 \\ 3 \\ -1 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ 0 \end{bmatrix} \]Because the product yields the zero vector, \(\vec{u}\) is in \(\text{Nul } A\).
(III) Basis for a Subspace
Definition
A basis for a subspace \(H\) of \(\mathbb{R}^n\) is a linearly independent set in \(H\) that spans \(H\).
If \(A\) is an invertible \(n \times n\) matrix, the columns of \(A\) form a linearly independent set and span \(\mathbb{R}^n\) by the Invertible Matrix Theorem (I.M.T.). Therefore, the columns of \(A\) form a basis for \(\mathbb{R}^n\).
In particular, the columns of the identity matrix \(I_n\) form a basis for \(\mathbb{R}^n\). The set of vectors:
\[ \vec{e}_1 = \begin{bmatrix} 1 \\ 0 \\ \vdots \\ 0 \end{bmatrix}, \quad \vec{e}_2 = \begin{bmatrix} 0 \\ 1 \\ \vdots \\ 0 \end{bmatrix}, \quad \dots, \quad \vec{e}_n = \begin{bmatrix} 0 \\ 0 \\ \vdots \\ 1 \end{bmatrix} \]is called the standard basis for \(\mathbb{R}^n\).
Find a basis for the null space of a matrix \(A\). Then find a nonzero vector in the null space.
Solution: Suppose after row reducing a matrix \(A\), we obtain the following equations for the null space (\(A\vec{x} = \vec{0}\)):
\[ \begin{cases} x_1 = -5x_5 \\ x_2 = x_2 \text{ (free)} \\ x_3 = -3x_5 \\ x_4 = 0 \\ x_5 = x_5 \text{ (free)} \end{cases} \]Writing this in parametric vector form:
\[ \vec{x} = \begin{bmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ x_5 \end{bmatrix} = \begin{bmatrix} -5x_5 \\ x_2 \\ -3x_5 \\ 0 \\ x_5 \end{bmatrix} = x_2 \begin{bmatrix} 0 \\ 1 \\ 0 \\ 0 \\ 0 \end{bmatrix} + x_5 \begin{bmatrix} -5 \\ 0 \\ -3 \\ 0 \\ 1 \end{bmatrix} = x_2\vec{u} + x_5\vec{v} \]This expression represents all the linear combinations of \(\vec{u}\) and \(\vec{v}\), and they are linearly independent. So, \(\{\vec{u}, \vec{v}\}\) is a basis of \(\text{Nul } A\).
\(\vec{u}\) is a nonzero vector in \(\text{Nul } A\).
Note: \(\{\vec{u}, \vec{v}\}\) is not the only basis for \(\text{Nul } A\). For example, if \(\vec{w} = \vec{u} + \vec{v}\), then \(\{\vec{u}, \vec{w}\}\) is also a basis for \(\text{Nul } A\) since \(A\vec{w} = \vec{0}\) and \(\{\vec{u}, \vec{w}\}\) is linearly independent.
Theorem 13
The pivot columns of a matrix \(A\) form a basis for \(\text{Col } A\).
(Warning: Use the pivot columns of the original matrix \(A\), not the columns of the echelon form of \(A\)).
Find a basis for \(\text{Col } A\) for the matrix in Example 3.
Solution: Let \(A = \begin{bmatrix} \vec{a}_1 & \vec{a}_2 & \vec{a}_3 & \vec{a}_4 & \vec{a}_5 \end{bmatrix}\). Suppose its Reduced Row Echelon Form (RREF) is \(B\):
\[ B = \begin{bmatrix} \vec{b}_1 & \vec{b}_2 & \vec{b}_3 & \vec{b}_4 & \vec{b}_5 \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 & 0 & 5 \\ 0 & 0 & 1 & 0 & 3 \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 0 \end{bmatrix} \]\(\vec{b}_1\), \(\vec{b}_3\), and \(\vec{b}_4\) are the pivot columns of \(B\).
Therefore, the corresponding columns \(\vec{a}_1\), \(\vec{a}_3\), and \(\vec{a}_4\) are the pivot columns of the original matrix \(A\).
By Theorem 13, \(\{\vec{a}_1, \vec{a}_3, \vec{a}_4\}\) forms a basis for \(\text{Col } A\).
In fact, we have:
\(\vec{b}_5 = 5\vec{b}_1 + 3\vec{b}_3 \implies \vec{a}_5 = 5\vec{a}_1 + 3\vec{a}_3\)
Row operations do not affect linear dependence relations among the columns of the matrix. So, \(\{\vec{a}_1, \vec{a}_3, \vec{a}_4\}\) spans \(\text{Col } A\) and is linearly independent.
Note: \(\{\vec{a}_1, \vec{a}_3, \vec{a}_4\}\) is not the only basis for \(\text{Col } A\). In fact, any 3 linearly independent vectors in \(\text{Col } A\) form a basis for \(\text{Col } A\). For example, \(\{\vec{a}_1, \vec{a}_3, \vec{a}_5\}\) also forms a basis for \(\text{Col } A\).
Remark
If \(B\) is the Reduced Row Echelon Form (RREF) of \(A\), then:
- \(\text{Nul } A = \text{Nul } B\) since \(A\vec{x} = \vec{0} \iff B\vec{x} = \vec{0}\).
- \(\text{Col } A \neq \text{Col } B\). For instance, \(\vec{a}_1 \in \text{Col } A\), but generally \(\vec{a}_1 \notin \text{Col } B\). The pivot columns of \(B\) (e.g., \(\{\vec{b}_1, \vec{b}_3, \vec{b}_4\}\)) span \(\text{Col } B\), but they do not span \(\text{Col } A\).
If \(A\) is a \(6 \times 6\) matrix and \(\text{Nul } A\) is not the zero subspace, what can you say about \(\text{Col } A\)?
Solution:
Since \(\text{Nul } A \neq \{\vec{0}\}\), the equation \(A\vec{x} = \vec{0}\) has nontrivial solutions.
\(A\) is a square matrix, so \(A\) is not invertible by the Invertible Matrix Theorem.
Therefore, the columns of \(A\) do not span \(\mathbb{R}^6\).
Conclusion: \(\text{Col } A\) is a subspace of \(\mathbb{R}^6\), but \(\text{Col } A \neq \mathbb{R}^6\).