Section 4.3 Linearly Independent Sets; Bases

(I) Linearly Independent Sets

Recall from Section 1.7:

An indexed set \(\{\vec{v}_1, \dots, \vec{v}_p\}\) in a vector space \(V\) is said to be linearly independent (or dependent) if the vector equation:

c_1\vec{v}_1 + \dots + c_p\vec{v}_p = \vec{0} \quad (*)

has only the trivial solution (independent) or has a nontrivial solution (dependent).

The equation \((*)\) is called a linear dependence relation if the set is linearly dependent.
For 2 vectors: simply check if one is a scalar multiple of the other.

Theorem 4

An indexed set \(\{\vec{v}_1, \dots, \vec{v}_p\}\) of two or more vectors, with \(\vec{v}_1 \neq \vec{0}\), is linearly dependent if and only if some \(\vec{v}_j\) (with \(j > 1\)) is a linear combination of the preceding vectors.

(This corresponds to Theorem 7 in Section 1.7).

The main difference between linear dependence in \(\mathbb{R}^n\) and in a general vector space is that the vectors cannot simply be made into columns of a matrix \(A\) in order to study the equation \(A\vec{x} = \vec{0}\). We must rely directly on the definition and on Theorem 4.

Example 1

Let \(M_1 = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix}\), \(M_2 = \begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix}\), and \(M_3 = \begin{bmatrix} 4 & -1 \\ 0 & 0 \end{bmatrix}\). Is \(\{M_1, M_2, M_3\}\) a linearly dependent set?

Solution: Yes. Since \(M_3 = 4M_1 - M_2\), one matrix is a linear combination of the others. Thus, \(\{M_1, M_2, M_3\}\) is a linearly dependent set.

Example 2

Show that the set \(\mathcal{B} = \{1 + t^2, t + t^2, 1 + 2t + t^2\}\) is linearly independent in \(\mathbb{P}_2\).

Solution: Set up the linear combination equal to the zero polynomial:

\[ c_1(1 + t^2) + c_2(t + t^2) + c_3(1 + 2t + t^2) = 0 \]

Group the terms by powers of \(t\):

\[ (c_1 + c_3) + (c_2 + 2c_3)t + (c_1 + c_2 + c_3)t^2 = 0 \]

This gives a system of equations:

\[ \begin{cases} c_1 + c_3 = 0 \\ c_2 + 2c_3 = 0 \\ c_1 + c_2 + c_3 = 0 \end{cases} \]

From the first equation, \(c_1 = -c_3\). From the second, \(c_2 = -2c_3\). Substituting into the third equation:

\[ (-c_3) + (-2c_3) + c_3 = 0 \implies -2c_3 = 0 \implies c_3 = 0 \]

If \(c_3 = 0\), then \(c_1 = 0\) and \(c_2 = 0\). Because the equation has only the trivial solution, \(\mathcal{B}\) is linearly independent in \(\mathbb{P}_2\). (As we will see, \(\mathcal{B}\) is a basis for \(\mathbb{P}_2\)).

(II) Bases

Definition

Let \(H\) be a subspace of a vector space \(V\). An indexed set of vectors \(\mathcal{B} = \{\vec{b}_1, \dots, \vec{b}_p\}\) in \(V\) is a basis for \(H\) if:

\(\mathcal{B}\) is a linearly independent set.
\(H = \text{Span}\{\vec{b}_1, \dots, \vec{b}_p\}\).

Note: This definition applies to the case when \(H = V\).

Standard basis for \(\mathbb{P}_n\): The set \(S = \{1, t, \dots, t^n\}\) is a basis for \(\mathbb{P}_n\).

Example 3

Let \(\vec{u}_1 = \begin{bmatrix} 3 \\ 0 \\ -6 \end{bmatrix}\), \(\vec{u}_2 = \begin{bmatrix} -4 \\ 1 \\ 7 \end{bmatrix}\), and \(\vec{u}_3 = \begin{bmatrix} -2 \\ 1 \\ 5 \end{bmatrix}\).

Determine if \(\{\vec{u}_1, \vec{u}_2, \vec{u}_3\}\) is a basis for \(\mathbb{R}^3\).

Solution: Form matrix \(A\) with these vectors as columns and row reduce:

\( A = \begin{bmatrix} 3 & -4 & -2 \\ 0 & 1 & 1 \\ -6 & 7 & 5 \end{bmatrix} \xrightarrow{R_3 = R_3 + 2R_1} \begin{bmatrix} 3 & -4 & -2 \\ 0 & 1 & 1 \\ 0 & -1 & 1 \end{bmatrix} \xrightarrow{R_3 = R_3 + R_2} \begin{bmatrix} 3 & -4 & -2 \\ 0 & 1 & 1 \\ 0 & 0 & 2 \end{bmatrix} \)

Because there is a pivot in every row and every column, matrix \(A\) is invertible. By the Invertible Matrix Theorem, the columns of \(A\) form a basis for \(\mathbb{R}^3\).

(III) Bases for Nul A, Col A and Row A

(Previously discussed in Section 2.8)

Recall from Section 4.2:

\(\text{Row } A\) is the set of all linear combinations of the row vectors. \(\text{Row } A = \text{Col } A^T\).
\(\text{Row } A\) is a subspace of \(\mathbb{R}^n\) if \(A\) is an \(m \times n\) matrix.

Theorem 7

If two matrices \(A\) and \(B\) are row equivalent, then \(\text{Row } A = \text{Row } B\).

If \(B\) is in row echelon form (REF), then the nonzero rows of \(B\) form a basis for \(\text{Row } A\) as well as \(\text{Row } B\) (but not \(A\)).

Theorem 7 says that row reduction does not change the row space of a matrix. But row reduction does change the column space.

Example 4

Find the bases for \(\text{Col } A\) and \(\text{Row } A\).

\[ A = \begin{bmatrix} -2 & -5 & 8 & 0 & -17 \\ 1 & 3 & -5 & 1 & 5 \\ 3 & 11 & -19 & 7 & 1 \\ 1 & 7 & -13 & 5 & -3 \end{bmatrix} \]

Suppose the row echelon form (REF) of \(A\) is \(B\):

\[ A \sim B = \begin{bmatrix} 1 & 3 & -5 & 1 & 5 \\ 0 & 1 & -2 & 2 & -7 \\ 0 & 0 & 0 & -4 & 20 \\ 0 & 0 & 0 & 0 & 0 \end{bmatrix} \]

Basis for Row A:

The nonzero row vectors in \(B\) form a basis for \(\text{Row } A\). These are:

\{(1, 3, -5, 1, 5), \ (0, 1, -2, 2, -7), \ (0, 0, 0, -4, 20)\}

Basis for Col A:

Identify the pivot columns of \(B\) (columns 1, 2, and 4). The corresponding columns from the original matrix \(A\) form the basis for \(\text{Col } A\):

\left\{ \begin{bmatrix} -2 \\ 1 \\ 3 \\ 1 \end{bmatrix}, \begin{bmatrix} -5 \\ 3 \\ 11 \\ 7 \end{bmatrix}, \begin{bmatrix} 0 \\ 1 \\ 7 \\ 5 \end{bmatrix} \right\}

Warning: Row Space vs. Column Space

Row operations may change the linear dependence relations among the rows of a matrix.

Although the first 3 rows of \(B\) in Example 4 are linearly independent, it is wrong to conclude that the first 3 rows of \(A\) are linearly independent. In fact, for matrix \(A\), \(\vec{r}_3 = 2\vec{r}_1 + 7\vec{r}_2\).

(IV) The Spanning Set Theorem

A basis is an "efficient" spanning set that contains no unnecessary vectors. In fact, a basis can be constructed from a spanning set by discarding unneeded vectors.

Theorem 5 (The Spanning Set Theorem)

Let \(S = \{\vec{v}_1, \dots, \vec{v}_p\}\) be a set in \(V\), and \(H = \text{Span}\{\vec{v}_1, \dots, \vec{v}_p\}\).

If one of the vectors in \(S\) — say \(\vec{v}_k\) — is a linear combination of the remaining vectors in \(S\), then the set formed from \(S\) by removing \(\vec{v}_k\) still spans \(H\).
If \(H \neq \{\vec{0}\}\), some subset of \(S\) is a basis for \(H\).

Two views of a basis:

A basis is a spanning set that's as small as possible.
A basis is a linearly independent set that's as large as possible.

Example 5

The following 3 sets in \(\mathbb{R}^3\) show how a linearly independent set can be enlarged to a basis and how further enlargement destroys the linear independence of the set. Also, a spanning set can be shrunk to a basis, but further shrinking destroys the spanning property.

Linearly independent, but doesn't span \(\mathbb{R}^3\): \[ \left\{ \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}, \begin{bmatrix} 2 \\ 3 \\ 0 \end{bmatrix} \right\} \]
A basis for \(\mathbb{R}^3\) (Linearly independent AND spans): \[ \left\{ \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}, \begin{bmatrix} 2 \\ 3 \\ 0 \end{bmatrix}, \begin{bmatrix} 4 \\ 5 \\ 6 \end{bmatrix} \right\} \]
Spans \(\mathbb{R}^3\), but is not linearly independent: \[ \left\{ \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}, \begin{bmatrix} 2 \\ 3 \\ 0 \end{bmatrix}, \begin{bmatrix} 4 \\ 5 \\ 6 \end{bmatrix}, \begin{bmatrix} 7 \\ 8 \\ 9 \end{bmatrix} \right\} \]