Introduction

A fundamental idea in linear algebra is to view a linear combination as the product of a matrix and a vector, as defined below.

(I) Definition of \(A\vec{x}\)

Definition

If \(A\) is an \(m \times n\) matrix with columns \(\vec{a}_1, \dots, \vec{a}_n\) and if \(\vec{x}\) is in \(\mathbb{R}^n\), then the product \(A\vec{x}\) is the linear combination of the columns of \(A\) using the corresponding entries in \(\vec{x}\) as weights; that is:

\[ A\vec{x} = \begin{bmatrix} \vec{a}_1 & \dots & \vec{a}_n \end{bmatrix} \begin{bmatrix} x_1 \\ \vdots \\ x_n \end{bmatrix} = x_1\vec{a}_1 + x_2\vec{a}_2 + \dots + x_n\vec{a}_n \]

Note: The number of columns of \(A\) must equal the number of entries in \(\vec{x}\).

Example 1

Write the following linear system as a vector equation first, and then as a matrix equation.

\[ \begin{cases} 2x_1 - 2x_3 = 5 \\ -x_1 + 4x_2 - 7x_3 = 6 \end{cases} \]

Vector Equation:

\[ x_1 \begin{bmatrix} 2 \\ -1 \end{bmatrix} + x_2 \begin{bmatrix} 0 \\ 4 \end{bmatrix} + x_3 \begin{bmatrix} -2 \\ -7 \end{bmatrix} = \begin{bmatrix} 5 \\ 6 \end{bmatrix} \]

Matrix Equation:

\[ \begin{bmatrix} 2 & 0 & -2 \\ -1 & 4 & -7 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} = \begin{bmatrix} 5 \\ 6 \end{bmatrix} \]

We can establish the following equivalency for an \(m \times n\) matrix \(A = \begin{bmatrix} \vec{a}_1 & \dots & \vec{a}_n \end{bmatrix}\), a vector \(\vec{x} = \begin{bmatrix} x_1 \\ \vdots \\ x_n \end{bmatrix}\), and a vector \(\vec{b} = \begin{bmatrix} b_1 \\ \vdots \\ b_m \end{bmatrix}\):

\[ A\vec{x} = \vec{b} \iff x_1\vec{a}_1 + \dots + x_n\vec{a}_n = \vec{b} \]

Both of these share the same augmented matrix: \([\begin{matrix} \vec{a}_1 & \dots & \vec{a}_n & | & \vec{b} \end{matrix}]\).

Theorem 3: Equivalence of Forms

The following three equations/systems have the exact same solution set (they are logically equivalent):

  1. The matrix equation \(A\vec{x} = \vec{b}\)
  2. The vector equation \(x_1\vec{a}_1 + x_2\vec{a}_2 + \dots + x_n\vec{a}_n = \vec{b}\)
  3. The linear system whose augmented matrix is \([\begin{matrix} \vec{a}_1 & \dots & \vec{a}_n & | & \vec{b} \end{matrix}]\)

They are all solved in the same way—by row reducing the augmented matrix.

(II) Existence of Solutions

For a matrix \(A = \begin{bmatrix} \vec{a}_1 & \dots & \vec{a}_n \end{bmatrix}\), we can ask several equivalent questions:

Example 2

Let \(A = \begin{bmatrix} 1 & -3 & -4 \\ -3 & 2 & 6 \\ 5 & -1 & -8 \end{bmatrix}\) and \(\vec{b} = \begin{bmatrix} b_1 \\ b_2 \\ b_3 \end{bmatrix}\).

Is \(A\vec{x} = \vec{b}\) consistent for all possible \(b_1, b_2, \text{ and } b_3\)?

Solution: Form the augmented matrix and reduce it to Row Echelon Form (REF):

\( \begin{aligned} \begin{bmatrix} 1 & -3 & -4 & b_1 \\ -3 & 2 & 6 & b_2 \\ 5 & -1 & -8 & b_3 \end{bmatrix} &\xrightarrow{} \begin{bmatrix} 1 & -3 & -4 & b_1 \\ 0 & -7 & -6 & 3b_1 + b_2 \\ 0 & 14 & 12 & -5b_1 + b_3 \end{bmatrix} \\ &\xrightarrow{} \begin{bmatrix} 1 & -3 & -4 & b_1 \\ 0 & -7 & -6 & 3b_1 + b_2 \\ 0 & 0 & 0 & b_1 + 2b_2 + b_3 \end{bmatrix} \end{aligned} \)

The system is inconsistent if the last row translates to \(0 = \text{nonzero}\). That is, it is inconsistent if \[ b_1 + 2b_2 + b_3 \neq 0 \]

Therefore, \(A\vec{x} = \vec{b}\) is not always consistent because some choices of \(\vec{b}\) can make \(b_1 + 2b_2 + b_3\) nonzero. It fails to be consistent because the REF of \(A\) has a row of zeros. If \(A\) had a pivot in all three rows, we would not care about the calculations in the augmented column because, in that case, the echelon form of the augmented matrix couldn't have a row such as \([\begin{matrix} 0 & \dots & 0 & | & \text{nonzero} \end{matrix}]\).

Definition: Span of \(\mathbb{R}^m\)

A set of vectors \(\{\vec{v}_1, \dots, \vec{v}_p\}\) in \(\mathbb{R}^m\) spans \(\mathbb{R}^m\) if every vector in \(\mathbb{R}^m\) is a linear combination of \(\vec{v}_1, \dots, \vec{v}_p\).

That is, \(\text{Span}\{\vec{v}_1, \dots, \vec{v}_p\} = \mathbb{R}^m\).

Theorem 4

Let \(A\) be an \(m \times n\) matrix. Then the following statements are logically equivalent (i.e., if one is true, they are all true):

  1. For each \(\vec{b}\) in \(\mathbb{R}^m\), the equation \(A\vec{x} = \vec{b}\) has a solution.
  2. Each \(\vec{b}\) in \(\mathbb{R}^m\) is a linear combination of the columns of \(A\).
  3. The columns of \(A\) span \(\mathbb{R}^m\).
  4. \(A\) has a pivot position in every row.

Warning: The matrix \(A\) in Theorem 4 is a coefficient matrix, not an augmented matrix. If the augmented matrix \([A \ | \ \vec{b}]\) has a pivot in every row, then \(A\vec{x} = \vec{b}\) may or may not be consistent (e.g., the pivot could be in the last column).

Example 3

Let the matrix \(A\) be reduced to its Row Echelon Form (REF):

\[ A \xrightarrow{\text{REF}} \begin{bmatrix} 1 & 3 & 0 & 3 \\ 0 & 2 & -1 & 4 \\ 0 & 0 & 0 & 6 \\ 0 & 0 & 0 & 0 \end{bmatrix} \]
  1. How many rows of \(A\) contain a pivot position?
    Answer: 3 rows.
  2. Does the equation \(A\vec{x} = \vec{b}\) have a solution for all \(\vec{b}\) in \(\mathbb{R}^4\)?
    Answer: No! (Because there is a row of all zeros).
  3. Can each vector in \(\mathbb{R}^4\) be written as a linear combination of the columns of \(A\)?
    Answer: No!
  4. Do the columns of \(A\) span \(\mathbb{R}^4\)?
    Answer: No! (By Theorem 4).

(III) Computation of \(A\vec{x}\)

There are two ways to compute the product \(A\vec{x}\): using the Definition (linear combination of columns) and the Row-Vector Rule.

Row-Vector Rule for Computing \(A\vec{x}\)

If the product \(A\vec{x}\) is defined (number of columns of \(A\) equals the number of entries in \(\vec{x}\)), then the \(i\)-th entry in \(A\vec{x}\) is the sum of the products of corresponding entries from Row \(i\) of \(A\) and the vector \(\vec{x}\).

Example 4

Compute the product: \( \begin{bmatrix} 2 & -3 \\ 8 & 0 \\ -5 & 2 \end{bmatrix} \begin{bmatrix} 4 \\ 7 \end{bmatrix} \)

Method 1: Using the Definition (Linear Combination)

\( \begin{aligned} 4 \begin{bmatrix} 2 \\ 8 \\ -5 \end{bmatrix} + 7 \begin{bmatrix} -3 \\ 0 \\ 2 \end{bmatrix} &= \begin{bmatrix} 4(2) + 7(-3) \\ 4(8) + 7(0) \\ 4(-5) + 7(2) \end{bmatrix} = \begin{bmatrix} 8 - 21 \\ 32 + 0 \\ -20 + 14 \end{bmatrix} = \begin{bmatrix} -13 \\ 32 \\ -6 \end{bmatrix} \end{aligned} \)

Method 2: Using the Row-Vector Rule

\( \begin{aligned} \begin{bmatrix} 2(4) + (-3)(7) \\ 8(4) + 0(7) \\ (-5)(4) + 2(7) \end{bmatrix} &= \begin{bmatrix} -13 \\ 32 \\ -6 \end{bmatrix} \end{aligned} \)

Identity Matrix

The \(n \times n\) Identity Matrix, denoted \(I_n\), has 1s on the main diagonal and 0s everywhere else:

\[ I_n = \begin{bmatrix} 1 & 0 & \dots & 0 \\ 0 & 1 & \dots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \dots & 1 \end{bmatrix}_{n \times n} \]

Property: \(I_n \vec{x} = \vec{x}\) for every \(\vec{x}\) in \(\mathbb{R}^n\).

Theorem 5: Properties of the Matrix-Vector Product \(A\vec{x}\)

If \(A\) is an \(m \times n\) matrix, \(\vec{u}\) and \(\vec{v}\) are vectors in \(\mathbb{R}^n\), and \(c\) is a scalar, then:

  1. \( A(\vec{u} + \vec{v}) = A\vec{u} + A\vec{v} \)
  2. \( A(c\vec{u}) = c(A\vec{u}) \)

(This can be proved directly by the definition of \(A\vec{x}\). Theorem 5 will be used extensively later.)