PAGE 2

6.1 Inner Product, Length, and Orthogonality

If \( \vec{u}, \vec{v} \) are in \( \mathbb{R}^n \)

\[ \vec{u} = \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix} \]
\[ \vec{v} = \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix} \]

the inner product (or dot product) of \( \vec{u} \) and \( \vec{v} \) is

\[ \vec{u} \cdot \vec{v} = \vec{u}^T \vec{v} = [u_1, u_2, \dots, u_n] \begin{bmatrix} v_1 \\ \vdots \\ v_n \end{bmatrix} \]\[ = u_1 v_1 + u_2 v_2 + \dots + u_n v_n \quad \text{a } \underline{\text{scalar}} \]
\[ \vec{u} = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix} \quad \vec{v} = \begin{bmatrix} 6 \\ 5 \\ 4 \end{bmatrix} \quad \vec{u} \cdot \vec{v} = [1 \ 2 \ 3] \begin{bmatrix} 6 \\ 5 \\ 4 \end{bmatrix} = 6 + 10 + 12 = 28 \]

note \( \vec{u} \cdot \vec{v} = \vec{v} \cdot \vec{u} \)

properties:

\[ \vec{u} = \begin{bmatrix} 1 \\ 2 \end{bmatrix} \quad \vec{v} = \begin{bmatrix} 3 \\ 4 \end{bmatrix} \quad \vec{w} = \begin{bmatrix} 5 \\ 6 \end{bmatrix} \quad c = 7 \]
\[ (\vec{u} + \vec{v}) \cdot \vec{w} = \begin{bmatrix} 4 \\ 6 \end{bmatrix}^T \begin{bmatrix} 5 \\ 6 \end{bmatrix} = [4 \ 6] \begin{bmatrix} 5 \\ 6 \end{bmatrix} = 56 \]\[ = \vec{u} \cdot \vec{w} + \vec{v} \cdot \vec{w} = [1 \ 2] \begin{bmatrix} 5 \\ 6 \end{bmatrix} + [3 \ 4] \begin{bmatrix} 5 \\ 6 \end{bmatrix} = 17 + 39 = 56 \]
PAGE 3

Vector Properties and Norms

\[ (c\vec{u}) \cdot \vec{v} = \begin{bmatrix} 7 & 14 \end{bmatrix} \begin{bmatrix} 3 \\ 4 \end{bmatrix} = 21 + 56 = 77 \]\[ = c(\vec{u} \cdot \vec{v}) = 7 \begin{bmatrix} 1 & 2 \end{bmatrix} \begin{bmatrix} 3 \\ 4 \end{bmatrix} = 7(3 + 8) = 77 \]

Consider the vector \( \vec{u} = \begin{bmatrix} 1 \\ 2 \end{bmatrix} \). The dot product of the vector with itself is:

\[ \vec{u} \cdot \vec{u} = \begin{bmatrix} 1 & 2 \end{bmatrix} \begin{bmatrix} 1 \\ 2 \end{bmatrix} = 1 + 4 = 5 \]

The length of the vector is calculated as:

\[ \text{length} = \sqrt{1^2 + 2^2} = \sqrt{5} \]
A 2D coordinate graph showing vector u from the origin to point (1, 2) with length square root of 5.

So, \[ \vec{u} \cdot \vec{u} = \|\vec{u}\|^2 \] for \( \vec{u} \) in \( \mathbb{R}^n \).

length or norm of \( \vec{u} \)

Unit Vectors

A vector is called a unit vector if its length is 1.

So a unit vector in the same direction as \( \vec{u} \) is:

\[ \frac{\vec{u}}{\|\vec{u}\|} = \begin{bmatrix} 1/\sqrt{5} \\ 2/\sqrt{5} \end{bmatrix} \]
PAGE 4

In \( \mathbb{R}^1 \), the distance between 2 numbers is \( |b - a| = |a - b| \).

A number line showing points a and b with the distance between them labeled as |b-a| = |a-b|.

Extend to \( \mathbb{R}^n \)

Example: \( \mathbb{R}^3 \)

Distance between 2 vectors is:

\[ \text{dist}(\vec{u}, \vec{v}) = \|\vec{v} - \vec{u}\| = \|\vec{u} - \vec{v}\| \]
Vector triangle showing vectors u, v, and the difference vector v minus u connecting their tips.
Vector triangle showing vectors u, v, and the difference vector u minus v connecting their tips.
PAGE 5
\[ \vec{u} = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix} \quad \vec{v} = \begin{bmatrix} -1 \\ -2 \\ -3 \end{bmatrix} \]\[ \text{dist}(\vec{u}, \vec{v}) = \| \vec{u} - \vec{v} \| = \| \vec{v} - \vec{u} \| \]\[ \vec{u} - \vec{v} = \begin{bmatrix} 2 \\ 4 \\ 6 \end{bmatrix} \quad \| \vec{u} - \vec{v} \| = \sqrt{84} \]

orthogonality

\(\vec{v}\) and \(\vec{u}\) are orthogonal if

\[ \| \vec{v} - \vec{u} \|^2 = \| \vec{v} + \vec{u} \|^2 \]
Coordinate graph with y and x axes showing vectors \vec{v}, -\vec{u}, and dashed lines for magnitudes.
\[ (\vec{v} - \vec{u}) \cdot (\vec{v} - \vec{u}) = (\vec{v} + \vec{u}) \cdot (\vec{v} + \vec{u}) \]\[ \vec{v} \cdot \vec{v} - \vec{v} \cdot \vec{u} - \vec{u} \cdot \vec{v} + \vec{u} \cdot \vec{u} = \vec{v} \cdot \vec{v} + \vec{v} \cdot \vec{u} + \vec{u} \cdot \vec{v} + \vec{u} \cdot \vec{u} \]\[ -2 \vec{u} \cdot \vec{v} = 2 \vec{u} \cdot \vec{v} \]\[ -\vec{u} \cdot \vec{v} = \vec{u} \cdot \vec{v} \]

true only if

\( \vec{u} \cdot \vec{v} = 0 \)

means \( \vec{u} \perp \vec{v} \)

PAGE 6

If \(W\) is a subspace of \(\mathbb{R}^n\). Then all vectors orthogonal to every vector in \(W\) is called the orthogonal complement of \(W\), written as \(W^{\perp}\) (read: "\(W\) perpendicular" or "\(W\) perp").

\(W^{\perp}\) is also a subspace of \(\mathbb{R}^n\)

example

The \(z\)-axis is the orthogonal complement of the subspace that is the \(xy\)-plane.

this is the same as

\[ \underbrace{\begin{bmatrix} 0 & 0 & 1 \end{bmatrix}}_{\text{unit vector on } z\text{-axis}} \underbrace{\begin{bmatrix} x \\ y \\ z \end{bmatrix}}_{\text{vectors in } \mathbb{R}^3} = 0 \]

look it as \(A\vec{x} = \vec{0}\)

this means the solution \(\vec{x}\) is in the nullspace of \(A\), \(\dim \text{Nul}(A) = 2\) (\(xy\)-plane)

\( \Rightarrow (\text{Row } A)^{\perp} = \text{Nul } A \)
PAGE 7

Likewise, \( (\text{Col } A)^{\perp} = \text{Nul } A^T \)

Coordinate graph showing vectors u and v in the first and second quadrants with angle theta and distance vector u-v.

Law of Cosines

\[ \|\vec{u} - \vec{v}\|^2 = \|\vec{v}\|^2 + \|\vec{u}\|^2 - 2\|\vec{u}\|\|\vec{v}\| \cos \theta \]
\[ \begin{aligned} \|\vec{u}\|\|\vec{v}\| \cos \theta &= -\frac{1}{2} \left( \|\vec{u} - \vec{v}\|^2 - \|\vec{v}\|^2 - \|\vec{u}\|^2 \right) \\ &= \frac{1}{2} \left( \|\vec{v}\|^2 + \|\vec{u}\|^2 - \|\vec{u} - \vec{v}\|^2 \right) \\ &= \frac{1}{2} \left( v_1^2 + v_2^2 + u_1^2 + u_2^2 - (u_1 - v_1)^2 - (u_2 - v_2)^2 \right) \\ &= \frac{1}{2} ( v_1^2 + v_2^2 + u_1^2 + u_2^2 - u_1^2 + 2u_1v_1 - v_1^2 \\ &\quad - u_2^2 + 2u_2v_2 - v_2^2 ) \end{aligned} \]
PAGE 8
\[ \|\vec{u}\|\|\vec{v}\| \cos \theta = u_1v_1 + u_2v_2 = \vec{u} \cdot \vec{v} \]

so,

\[ \|\vec{u}\|\|\vec{v}\| \cos \theta = \vec{u} \cdot \vec{v} \]