Column Space (Col(A)) & Null Space (Null(A))

\( m \times n \) Linear System

  1. \( a_{11}x_1 + a_{12}x_2 + \cdots + a_{1n}x_n = b_1 \)
  2. \( a_{21}x_1 + a_{22}x_2 + \cdots + a_{2n}x_n = b_2 \)
  3. \( a_{m1}x_1 + a_{m2}x_2 + \cdots + a_{mn}x_n = b_m \)

unknowns: \( x_1, x_2, x_3, \ldots, x_n \)

Column Space (Col(A)) & Null Space (Null(A))

\( m \times n \) linear system, in Matrix form :

\[ \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ & & a_{ij} & \\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix} = \begin{bmatrix} b_1 \\ b_2 \\ \vdots \\ b_m \end{bmatrix} \]

\( A^{(m \times n)} \), \( \vec{x} \in \mathbb{R}^n \), \( \vec{b} \in \mathbb{R}^m \)

first row of \( AX \): \( a_{11}x_1 + a_{12}x_2 + \cdots + a_{1n}x_n \)

second row of \( AX \): \( a_{21}x_1 + a_{22}x_2 + \cdots + a_{2n}x_n \)

\(i\)-th row of \( AX \): \( a_{i1}x_1 + a_{i2}x_2 + \cdots + a_{in}x_n \)

\(m\)-th row of \( AX \): \( a_{m1}x_1 + a_{m2}x_2 + \cdots + a_{mn}x_n \)

Properties of the "Notation" \( A\vec{X} \)

matrix multiply by a vector

\[ A\vec{X} = \vec{b} \]

\( A^{(m \times n)} \), \( \vec{X} \in \mathbb{R}^n \), \( \vec{b} \in \mathbb{R}^m \)

Linearity Property of Matrix Multiplication

  1. \( A(\vec{X} + \vec{Y}) = A\vec{X} + A\vec{Y} \)
  2. \( A(\alpha \vec{X}) = \alpha A\vec{X} \)
  3. \( (A + B)(\vec{X}) = A\vec{X} + B\vec{X} \)
  4. \( (\alpha A)(\vec{X}) = \alpha(A\vec{X}) \)

Properties of the "Notation" \( A\vec{X} \)

matrix multiplication by a vector

\[ A\vec{X} = \vec{b} \]

\( A^{(m \times n)} \), \( \vec{X} \in \mathbb{R}^n \), \( \vec{b} \in \mathbb{R}^m \)

Linearity Property of Matrix Multiplication

  1. \( A(\alpha \vec{X} + \beta \vec{Y}) = \alpha (A\vec{X}) + \beta (A\vec{Y}) \)
  2. \( (\alpha A + \beta B)(\vec{X}) = \alpha (A\vec{X}) + \beta (B\vec{X}) \)

Column Space of A and Solution of \( A\vec{X} = \vec{b}\)

\( a_{11}x_1 + a_{12}x_2 + \cdots + a_{1n}x_n = b_1 \)

\( a_{21}x_1 + a_{22}x_2 + \cdots + a_{2n}x_n = b_2 \)

\( \cdots \)

\( a_{m1}x_1 + a_{m2}x_2 + \cdots + a_{mn}x_n = b_m \)

\[ x_1 \begin{bmatrix} a_{11} \\ a_{21} \\ \vdots \\ a_{m1} \end{bmatrix} + x_2 \begin{bmatrix} a_{12} \\ a_{22} \\ \vdots \\ a_{m2} \end{bmatrix} + \cdots + x_n \begin{bmatrix} a_{1n} \\ a_{2n} \\ \vdots \\ a_{mn} \end{bmatrix} = \begin{bmatrix} b_1 \\ b_2 \\ \vdots \\ b_m \end{bmatrix} \]

columns of A

Column Space of A and Solution of \( A\vec{X} = \vec{b} \)

  1. \[ A\vec{X} = \vec{b} \iff \vec{b} \text{ can be written as lin. comb. of columns of } A \]

  2. Column Space of A = Span \(\{\)columns of A\(\} = \text{Col}(A) \)

  3. \[ A\vec{X} = \vec{b} \iff \vec{b} \in \text{Col}(A) \]

Column Space of A and Solution of \( A\vec{X} = \vec{b} \)

Forward picture: mapping X in R^n through A into AX in Col(A) within R^m
Figure 1
A hand-drawn diagram labeled 'Forward picture' illustrating the linear transformation given by matrix A. On the left, a blue coordinate frame with axes labeled x1, x2, ..., xn represents the domain, an orange-highlighted region labeled R^n. A red vector labeled X emerges from the origin of this frame. A large red curved arrow labeled A points from the domain to the codomain on the right. On the right, another blue coordinate frame with axes labeled y1, y2, ..., ym represents the codomain, an orange-highlighted region labeled R^m. Inside this frame is a red parallelogram (a plane) labeled Col(A), and a red vector labeled AX lies within this plane. The diagram shows that A maps X in R^n to AX, which lies in the column space Col(A) inside R^m.

\( \vec{X} \in \mathbb{R}^n \longrightarrow A\vec{X} \in \text{Col}(A)\subseteq \mathbb{R}^m \)

Column Space of A and Solution of \( A\vec{X} = \vec{b} \)

Backward picture

Backward picture: mapping b in Col(A) within R^m back through A to X in R^n
Figure 2
A hand-drawn 'Backward picture' illustrating the map A between two coordinate spaces. On the left is a coordinate system labeled R^n with axes x_1, x_2, x_n, containing a point labeled X. On the right is a coordinate system labeled R^m with axes y_1, y_2, y_m, containing a red parallelogram (a plane) labeled Col(A) with a point b marked inside and an arrow pointing to it. A long red curved arrow, labeled A, runs from the point b in Col(A) back to the point X, indicating the backward direction of the mapping. A boxed statement below reads: Given b in Col(A), find X in R^n s.t. AX = b.

Given \( \vec{b} \in \text{Col}(A) \), find \( \vec{X} \in \mathbb{R}^n \) s.t. \( A\vec{X} = \vec{b} \)

Column Space of A and Solution of \( A\vec{X} = \vec{b} \)

Backward picture

Backward picture: b in Col(A) within R^m mapped back through A to X in R^n
Figure 3
A hand-drawn backward-picture diagram in red and blue. On the right is a domain space labeled R^n with two blue coordinate axes (x_1 and x_2) and a red point labeled X with a question mark. On the left is a codomain space labeled R^m with blue coordinate axes (y_1, y_2, and y_m) containing a red parallelogram plane labeled Col(A); a point b lies in this plane, labeled b in Col(A). A large red curved arrow labeled A points from the b point in Col(A) back to the unknown X in R^n, representing the search for X such that AX = b.

Note

If \( \vec{b} \notin \text{Col}(A) \), then there is no \( \vec{X} \) s.t. \( A\vec{X} = \vec{b} \)

Homogeneous vs Inhomogeneous System

Homogeneous System (H)

\[ A\vec{X} = \vec{0} \]\[ \left[\begin{array}{c|c} A & \begin{matrix} 0 \\ 0 \\ \vdots \\ 0 \end{matrix} \end{array}\right] \longrightarrow \longrightarrow \left[\begin{array}{c|c} R & \begin{matrix} 0 \\ 0 \\ \vdots \\ 0 \end{matrix} \end{array}\right] \]

Solution is written as :

\[ \vec{X}_h = s_1 \vec{V}_1 + s_2 \vec{V}_2 + \cdots + s_k \vec{V}_k \]

\( s_i \) — free variables

\( \vec{V}_i \) — spanning vectors

\( \text{Null Space of } A \ (\text{Null}(A)) = \{ \text{Solution of } A\vec{X} = \vec{0} \} \)

Homogeneous vs Inhomogeneous System

Homogeneous System (H)

\[ A\vec{X} = \vec{0} \]\[ \left[\begin{array}{c|c} A & \begin{matrix} 0 \\ 0 \\ \vdots \\ 0 \end{matrix} \end{array}\right] \longrightarrow \longrightarrow \left[\begin{array}{c|c} R & \begin{matrix} 0 \\ 0 \\ \vdots \\ 0 \end{matrix} \end{array}\right] \]

Solution is written as :

\[ \vec{X}_h = s_1 \vec{V}_1 + s_2 \vec{V}_2 + \cdots + s_k \vec{V}_k \]

\( s_i \) — free variables

\( \vec{V}_i \) — spanning vectors

Null Space of A \( (\text{Null}(A)) = \) Span \( \{ \vec{V}_1, \vec{V}_2, \cdots \vec{V}_k \} \)

Homogeneous vs Inhomogeneous System

Diagram mapping X in Null(A) in R^n through A to the origin in R^m, illustrating AX = 0
Figure 4
A domain-and-codomain diagram illustrating the homogeneous system. On the right (labeled in orange) is the domain space R^n, drawn in blue axes with a red parallelogram representing a subspace; the red label reads X in Null(A) with an arrow pointing to a vector X lying in that subspace plane. On the left (labeled in orange) is the codomain space R^m, drawn with blue axes and a blue arrow. A red curved arrow labeled A maps from the vector X in the domain to a red dot at the origin of R^m. The red label AX = 0 marks this mapping to the zero vector, showing that vectors in the null space are sent to zero.

Homogeneous vs Inhomogeneous System

Inhomogeneous System (I)

\[ A\vec{X} = \vec{b} \] \[ \left[\begin{array}{c|c} A & \vec{b} \end{array}\right] \longrightarrow \longrightarrow \left[\begin{array}{c|c} R & \begin{matrix} * \\ * \\ * \\ * \end{matrix} \end{array}\right] \]

Solution is written as :

\[ \vec{X} = \vec{p} + s_1 \vec{V}_1 + s_2 \vec{V}_2 + \cdots + s_k \vec{V}_k \]

translation vector: \(\vec{p}\)

spanning vectors: \( \vec{V}_1, \vec{V}_2, \ldots, \vec{V}_k\)

Homogeneous vs Inhomogeneous System

Inhomogeneous System (I)

\[ A\vec{X} = \vec{b} \]\[ \left[\begin{array}{c|c} A & \vec{b} \end{array}\right] \longrightarrow \longrightarrow \left[\begin{array}{c|c} R & \begin{matrix} * \\ * \\ * \\ * \end{matrix} \end{array}\right] \]

Solution is written as :

\[ \vec{X} = \vec{p} + \text{Null}(A) \]

Shifted Null Space

Homogeneous vs Inhomogeneous System

Diagram of R^n domain and R^m codomain with a shifted null space plane mapping to b in Col(A)
Figure 5
A hand-drawn diagram illustrating an inhomogeneous system. On the right (domain side, labeled R^n) there are blue coordinate axes with a red slanted parallelogram representing a shifted null space; it is labeled in red 'Shifted Null Sp'. A point marked X sits on this plane, and a red vector labeled p points to it from the origin. A curved red arrow maps from this X in R^n across to the left (codomain side, labeled R^m), where blue coordinate axes appear and a red vector arrow points upward, labeled in red 'b in Col(A)'. Two orange-highlighted labels read 'R^n' and 'R^m' identifying the two spaces. The whole page is titled at top 'Homogeneous vs Inhomogeneous System', separated from the diagram by a red rule.

Solving \( m \times n \) Linear System

\[ A^{(m \times n)} X^{(n \times 1)} = b^{(m \times 1)} \]

Existence of Solution

\[ \vec{b} \in \text{Col}(A) \]

Uniqueness of Solution (if exists)

no free variables

\[ \text{Null}(A) = \{ \vec{0} \} \]

Solving \( m \times n \) Linear System

\[ A^{(m \times n)} X^{(n \times 1)} = b^{(m \times 1)} \]

(More unknown Theorem)

\[ n > m \]

If solution exists , then it cannot be unique, i.e. there must be some free. va

Echelon-form matrix sketch with m rows and n columns showing leading entries and free columns
Figure 6
A hand-drawn diagram in blue showing a wide matrix in echelon (staircase) form. The matrix is labeled with height m (number of rows) along the left and width n (number of columns) along the top and bottom. A descending staircase of short line segments with small arrows marks the leading (pivot) entries stepping down and to the right across the rows. Because the matrix is wider than tall (n greater than m), the staircase does not reach every column, illustrating that there are more columns than pivots, so free variables must exist.

Solving \( m \times n \) Linear System

\[ A^{(m \times n)} X^{(n \times 1)} = b^{(m \times 1)} \]

(More equation Theorem)

\[ m > n \]

There must be some \( b \) such that \( A\vec{X} = \vec{b} \) does not have a solution

Echelon-form matrix sketch with hatched leading region and a bottom row of zeros marked with two asterisks
Figure 7
A hand-drawn matrix outline in blue with a large square bracket on each side. Inside, a staircase pattern of hatched diagonal lines represents the leading (pivot) entries in the upper portion. The bottom row is drawn as a dashed line of zeros, ending in a row of 0 0 0 0 0. Two asterisk marks appear to the right of this bottom zero row, indicating the augmenting entries that make the system inconsistent when m > n.