(I) Definition and Linear Independence of Matrix Columns
Definition
An indexed set of vectors \(\{\vec{v}_1, \dots, \vec{v}_p\}\) in \(\mathbb{R}^n\) is said to be linearly independent if the vector equation:
has only the trivial solution.
The set \(\{\vec{v}_1, \dots, \vec{v}_p\}\) is said to be linearly dependent if there exists weights \(c_1, \dots, c_p\), not all zero, such that:
This is called a linear dependence relation among \(\vec{v}_1, \dots, \vec{v}_p\).
If \(A = \begin{bmatrix} \vec{v}_1 & \vec{v}_2 & \dots & \vec{v}_n \end{bmatrix}\) and \(\vec{x} = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}\), then:
Proposition
The columns of a matrix \(A\) are linearly independent
\(\iff\) The equation \(A\vec{x} = \vec{0}\) has only the trivial solution (no free variable)
\(\iff\) Every column is a pivot column.
Let \(\vec{v}_1 = \begin{bmatrix} 1 \\ -1 \\ 4 \end{bmatrix}\), \(\vec{v}_2 = \begin{bmatrix} 3 \\ -5 \\ 7 \end{bmatrix}\), and \(\vec{v}_3 = \begin{bmatrix} -1 \\ 5 \\ 6 \end{bmatrix}\).
- Determine if the set \(\{\vec{v}_1, \vec{v}_2, \vec{v}_3\}\) is linearly independent. (This is equivalent to: Determine if the columns of the matrix \(\begin{bmatrix} \vec{v}_1 & \vec{v}_2 & \vec{v}_3 \end{bmatrix}\) are linearly independent.)
- If possible, find a linear dependence relation among \(\vec{v}_1\), \(\vec{v}_2\), and \(\vec{v}_3\).
Solution: Construct the coefficient matrix and row reduce:
Writing the corresponding equations:
Because there is a free variable, the set \(\{\vec{v}_1, \vec{v}_2, \vec{v}_3\}\) is linearly dependent.
To find a specific linear dependence relation, let \(x_3 = 1\). Then \(x_1 = -5\), \(x_2 = 2\), and \(x_3 = 1\).
A linear dependence relation among \(\vec{v}_1, \vec{v}_2\), and \(\vec{v}_3\) is:
Note: This relation is not unique. We can find infinitely many linear dependence relations among \(\vec{v}_1\), \(\vec{v}_2\), and \(\vec{v}_3\) by choosing different values for the free variable.
Find the value(s) of \(h\) for which the vectors are linearly dependent:
Solution: Construct the coefficient matrix and row reduce to find the pivot columns:
Notice that \(x_3\) is a free variable. The equation \(x_1\vec{v}_1 + x_2\vec{v}_2 + x_3\vec{v}_3 = \vec{0}\) has a free variable and hence a nontrivial solution no matter what the value of \(h\) is.
So the vectors are linearly dependent for all values of \(h\).
(II) Sets of Vectors
Case 1: Sets of One Vector
The set \(\{\vec{v}_1\}\) is linearly independent if and only if \(x_1\vec{v}_1 = \vec{0}\) has only the trivial solution \(x_1 = 0\). This happens if and only if \(\vec{v}_1 \neq \vec{0}\).
- \(\{\vec{0}\}\) is always linearly dependent.
- A set of one vector \(\{\vec{v}_1\}\) is linearly independent if and only if \(\vec{v}_1 \neq \vec{0}\).
Case 2: Sets of Two Vectors \(\{\vec{v}_1, \vec{v}_2\}\)
The set is linearly dependent if and only if \(x_1\vec{v}_1 + x_2\vec{v}_2 = \vec{0}\) has a nontrivial solution.
- If \(x_1 \neq 0\), then \(\vec{v}_1 = -\frac{x_2}{x_1}\vec{v}_2\)
- If \(x_2 \neq 0\), then \(\vec{v}_2 = -\frac{x_1}{x_2}\vec{v}_1\)
- \(\{\vec{v}_1, \vec{v}_2\}\) is linearly dependent if at least one of the vectors is a multiple of the other.
- \(\{\vec{v}_1, \vec{v}_2\}\) is linearly independent if and only if neither of the vectors is a multiple of the other.
Note: We can always determine by inspection when a set of 2 vectors is linearly dependent.
Case 3: Sets of Two or More Vectors \(S = \{\vec{v}_1, \dots, \vec{v}_p\}\)
Theorem 7 (Characterization of Linearly Dependent Sets)
An indexed set \(S = \{\vec{v}_1, \dots, \vec{v}_p\}\) of two or more vectors is linearly dependent if and only if at least one of the vectors in \(S\) is a linear combination of the others.
In fact, if \(S\) is linearly dependent and \(\vec{v}_1 \neq \vec{0}\), then some \(\vec{v}_j\) (with \(j > 1\)) is a linear combination of the preceding vectors \(\vec{v}_1, \dots, \vec{v}_{j-1}\).
(Proof in textbook)
Let \(\vec{u} = \begin{bmatrix} 3 \\ 2 \\ -4 \end{bmatrix}\), \(\vec{v} = \begin{bmatrix} -6 \\ 1 \\ 7 \end{bmatrix}\), \(\vec{w} = \begin{bmatrix} 0 \\ -5 \\ 2 \end{bmatrix}\), and \(\vec{z} = \begin{bmatrix} 3 \\ 7 \\ -5 \end{bmatrix}\).
(a) Are the sets \(\{\vec{u}, \vec{v}\}\), \(\{\vec{u}, \vec{w}\}\), \(\{\vec{u}, \vec{z}\}\), \(\{\vec{v}, \vec{w}\}\), \(\{\vec{v}, \vec{z}\}\), \(\{\vec{w}, \vec{z}\}\) each linearly independent?
Yes. In each case, neither vector is a scalar multiple of the other.
(b) Does the answer to Part (a) imply that \(\{\vec{u}, \vec{v}, \vec{w}, \vec{z}\}\) is linearly independent?
The answer to Part (a) says nothing about the linear independence of the set of all four vectors.
(c) Is \(\vec{w}\) a linear combination of \(\vec{u}\), \(\vec{v}\), and \(\vec{z}\)?
Form the augmented matrix \(\begin{bmatrix} \vec{u} & \vec{v} & \vec{z} & | & \vec{w} \end{bmatrix}\):
The system is inconsistent. Therefore, \(\vec{w}\) is not a linear combination of \(\vec{u}\), \(\vec{v}\), and \(\vec{z}\).
Note: When testing for linear independence, it's usually a poor idea to check if one selected vector is a linear combination of the others.
(d) Is \(\{\vec{u}, \vec{v}, \vec{w}, \vec{z}\}\) linearly independent?
Form the coefficient matrix \(\begin{bmatrix} \vec{u} & \vec{v} & \vec{w} & \vec{z} \end{bmatrix}\):
\(x_4\) is a free variable. Therefore, \(\{\vec{u}, \vec{v}, \vec{w}, \vec{z}\}\) is linearly dependent. (Or use Theorem 8 below).
Warning: Theorem 7 doesn't say that every vector in a linearly dependent set is a linear combination of the preceding vectors. A vector in a linearly dependent set may fail to be a linear combination of the other vectors.
The next two theorems describe special cases in which the linear dependence of the set is automatic.
Theorem 8
If a set contains more vectors than there are entries in each vector, then the set is linearly dependent.
Proof:
Let \(A = \begin{bmatrix} \vec{v}_1 & \dots & \vec{v}_p \end{bmatrix}\).
If \(A\vec{x} = \vec{0}\) has more unknowns than the number of equations, then \(A\) has a larger number of columns than the number of rows.
There must be free variables, meaning nontrivial solutions exist.
\(\implies \{\vec{v}_1, \dots, \vec{v}_p\}\) is linearly dependent.
Theorem 9
Any set containing \(\vec{0}\) is linearly dependent.
Proof:
Suppose \(\vec{v}_1 = \vec{0}\).
Then \(1\cdot\vec{v}_1 + 0\cdot\vec{v}_2 + \dots + 0\cdot\vec{v}_p = \vec{0}\) is a valid linear dependence relation (since at least one weight, the first, is non-zero).
Determine by inspection if the given set is linearly dependent.
- \(\left\{ \begin{bmatrix} 3 \\ 2 \\ 0 \end{bmatrix}, \begin{bmatrix} 6 \\ 0 \\ 4 \end{bmatrix} \right\}\)
Answer: Linearly independent. (Neither is a scalar multiple of the other). - \(\left\{ \begin{bmatrix} 1 \\ 7 \\ 6 \end{bmatrix}, \begin{bmatrix} 2 \\ 0 \\ 9 \end{bmatrix}, \begin{bmatrix} 3 \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} 4 \\ 1 \\ 8 \end{bmatrix} \right\}\)
Answer: Linearly dependent by Theorem 8 (4 vectors in \(\mathbb{R}^3\)). - \(\left\{ \begin{bmatrix} 1 \\ 2 \\ 0 \\ 1 \end{bmatrix}, \begin{bmatrix} -1 \\ 5 \\ 3 \\ 2 \end{bmatrix}, \begin{bmatrix} 0 \\ 0 \\ 0 \\ 0 \end{bmatrix} \right\}\)
Answer: Linearly dependent by Theorem 9 (contains the zero vector).
(1) How many pivot columns must a \(5 \times 4\) matrix have if its columns are linearly independent? Why?
Solution: All 4 columns of the \(5 \times 4\) matrix must be pivot columns. Otherwise, \(A\vec{x} = \vec{0}\) would have a free variable, in which case the columns of \(A\) would be linearly dependent.
(2) How many pivot columns must a \(4 \times 5\) matrix \(A\) have if its columns span \(\mathbb{R}^4\)? Why?
Solution: If the matrix spans \(\mathbb{R}^4\), then \(A\) has a pivot in each row by Theorem 4. Since each pivot position is in a different column, \(A\) has 4 pivot columns.