PAGE 1

6.2 Orthogonal Sets

A set of vectors is an orthogonal set if the vectors are mutually orthogonal.

\[ \{ \vec{u}_1, \vec{u}_2, \dots, \vec{u}_p \} \text{ is orthogonal if } \vec{u}_i \cdot \vec{u}_j = 0 \text{ when } i \neq j \]

Example

\[ \left\{ \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} \right\} = \{ \vec{i}, \vec{j}, \vec{k} \} \]\[ \vec{i} \cdot \vec{j} = \vec{i} \cdot \vec{k} = \vec{j} \cdot \vec{k} = 0 \]

Example

The set

\[ \left\{ \begin{bmatrix} 1 \\ -4 \\ 1 \end{bmatrix}, \begin{bmatrix} 0 \\ 1 \\ 4 \end{bmatrix}, \begin{bmatrix} -17 \\ -4 \\ 1 \end{bmatrix} \right\} \text{ is an orthogonal set} \]

Verification via dot products:

\[ \begin{bmatrix} 1 & -4 & 1 \end{bmatrix} \begin{bmatrix} 0 \\ 1 \\ 4 \end{bmatrix} = 0 \]\[ \begin{bmatrix} 1 & -4 & 1 \end{bmatrix} \begin{bmatrix} -17 \\ -4 \\ 1 \end{bmatrix} = 0 \]\[ \begin{bmatrix} 0 & 1 & 4 \end{bmatrix} \begin{bmatrix} -17 \\ -4 \\ 1 \end{bmatrix} = 0 \]

PAGE 2

If \( S = \{ \vec{u}_1, \dots, \vec{u}_p \} \) is an orthogonal set of nonzero vectors in \( \mathbb{R}^n \), then \( S \) is linearly independent and a basis for the subspace spanned by \( S \).

Why?

Let \( S = \{ \vec{u}_1, \vec{u}_2, \vec{u}_3 \} \) be an orthogonal set (so \( \vec{u}_1 \cdot \vec{u}_2 = \vec{u}_1 \cdot \vec{u}_3 = \vec{u}_2 \cdot \vec{u}_3 = 0 \)).

Suppose \( \vec{0} = c_1 \vec{u}_1 + c_2 \vec{u}_2 + c_3 \vec{u}_3 \) for some scalars \( c_1, c_2, c_3 \).

\( \vec{0} \cdot \vec{u}_1 = c_1 \vec{u}_1 \cdot \vec{u}_1 + c_2 \vec{u}_2 \cdot \vec{u}_1 + c_3 \vec{u}_3 \cdot \vec{u}_1 \rightarrow c_1 = 0 \)

\( \vec{0} \cdot \vec{u}_2 = c_1 \vec{u}_1 \cdot \vec{u}_2 + c_2 \vec{u}_2 \cdot \vec{u}_2 + c_3 \vec{u}_3 \cdot \vec{u}_2 \rightarrow c_2 = 0 \)

\( \vec{0} \cdot \vec{u}_3 = c_1 \vec{u}_1 \cdot \vec{u}_3 + c_2 \vec{u}_2 \cdot \vec{u}_3 + c_3 \vec{u}_3 \cdot \vec{u}_3 \rightarrow c_3 = 0 \)

So, \( c_1 = c_2 = c_3 = 0 \) is the only way.

\[ \vec{0} = c_1 \vec{u}_1 + c_2 \vec{u}_2 + c_3 \vec{u}_3 \rightarrow \{ \vec{u}_1, \vec{u}_2, \vec{u}_3 \} \text{ is linearly indp.} \]

PAGE 3

Orthogonal Bases

If a basis is orthogonal, then it is an orthogonal basis.

Makes calculating the weights of a linear combo easy.

Example

Orthogonal basis: \( \left\{ \begin{bmatrix} 2 \\ -3 \end{bmatrix}, \begin{bmatrix} 6 \\ 4 \end{bmatrix} \right\} = \{ \vec{u}_1, \vec{u}_2 \} \)

\[ \vec{x} = \begin{bmatrix} 9 \\ -7 \end{bmatrix} \quad c_1 \begin{bmatrix} 2 \\ -3 \end{bmatrix} + c_2 \begin{bmatrix} 6 \\ 4 \end{bmatrix} = \begin{bmatrix} 9 \\ -7 \end{bmatrix} \]

\( c_1 = ? \quad c_2 = ? \)

"Old" way: form augmented matrix, then row reduce.

Another Way: Take Advantage of Orthogonality

\[ \vec{x} = c_1 \vec{u}_1 + c_2 \vec{u}_2 \]\[ \vec{x} \cdot \vec{u}_1 = c_1 \vec{u}_1 \cdot \vec{u}_1 + c_2 \vec{u}_2 \cdot \vec{u}_1 \rightarrow c_1 = \frac{\vec{x} \cdot \vec{u}_1}{\vec{u}_1 \cdot \vec{u}_1} \]\[ \vec{x} \cdot \vec{u}_2 = c_1 \vec{u}_1 \cdot \vec{u}_2 + c_2 \vec{u}_2 \cdot \vec{u}_2 \rightarrow c_2 = \frac{\vec{x} \cdot \vec{u}_2}{\vec{u}_2 \cdot \vec{u}_2} \]

Note: In the equations above, terms involving \( \vec{u}_2 \cdot \vec{u}_1 \) and \( \vec{u}_1 \cdot \vec{u}_2 \) are zero due to orthogonality.

PAGE 4

\[ c_1 = \frac{\begin{bmatrix} 9 & -7 \end{bmatrix} \begin{bmatrix} 2 \\ -3 \end{bmatrix}}{\begin{bmatrix} 2 & -3 \end{bmatrix} \begin{bmatrix} 2 \\ -3 \end{bmatrix}} = \frac{39}{13} = 3 \]\[ c_2 = \frac{\begin{bmatrix} 9 & -7 \end{bmatrix} \begin{bmatrix} 6 \\ 4 \end{bmatrix}}{\begin{bmatrix} 6 & 4 \end{bmatrix} \begin{bmatrix} 6 \\ 4 \end{bmatrix}} = \frac{26}{52} = \frac{1}{2} \]

Orthogonal Projections

The diagram illustrates the decomposition of a vector \( \vec{y} \) into two components relative to a vector \( \vec{u} \).

Component of \( \vec{y} \) orthogonal to \( \vec{u} \): denoted as \( \vec{z} \).
"Shadow" of \( \vec{y} \): the orthogonal projection of \( \vec{y} \) onto \( \vec{u} \), denoted as \( \hat{y} \) ("y hat").

Figure: Geometric diagram showing vector y projected onto vector u as y-hat, with orthogonal component z.

Derivation

\[ \hat{y} + \vec{z} = \vec{y} \quad \text{let } \hat{y} = \alpha \vec{u} \]\[ \text{then } \vec{z} = \vec{y} - \hat{y} = \vec{y} - \alpha \vec{u} \]\[ \vec{z} \cdot \vec{u} = 0 = (\vec{y} - \alpha \vec{u}) \cdot \vec{u} = \vec{y} \cdot \vec{u} - \alpha \vec{u} \cdot \vec{u} \]

PAGE 5

\[ \alpha = \frac{\vec{y} \cdot \vec{u}}{\vec{u} \cdot \vec{u}} \]

therefore,

\[ \hat{y} = \frac{\vec{y} \cdot \vec{u}}{\vec{u} \cdot \vec{u}} \vec{u} \]

orthogonal projection of \( \vec{y} \) onto subspace spanned by \( \vec{u} \)

Length of \( \vec{u} \) doesn't matter

let \( \vec{u} = c\vec{u} \)

\[ \hat{y} = \frac{\vec{y} \cdot c\vec{u}}{c\vec{u} \cdot c\vec{u}} c\vec{u} = \frac{\cancel{c^2} \vec{y} \cdot \vec{u}}{\cancel{c^2} \vec{u} \cdot \vec{u}} \vec{u} \]

Example

\( \vec{y} = \begin{bmatrix} 2 \\ 3 \end{bmatrix} \quad \vec{u} = \begin{bmatrix} 1 \\ -5 \end{bmatrix} \)

write \( \vec{y} \) as \( \vec{y} = \hat{y} + \vec{z} \)

Figure: Vector diagram showing vector y projected onto vector u, with orthogonal component z and projection y-hat.

\[ \hat{y} = \frac{\begin{bmatrix} 2 & 3 \end{bmatrix} \begin{bmatrix} 1 \\ -5 \end{bmatrix}}{\begin{bmatrix} 1 & -5 \end{bmatrix} \begin{bmatrix} 1 \\ -5 \end{bmatrix}} \begin{bmatrix} 1 \\ -5 \end{bmatrix} = \frac{-13}{26} \begin{bmatrix} 1 \\ -5 \end{bmatrix} = \begin{bmatrix} -1/2 \\ 5/2 \end{bmatrix} \]

\[ \vec{z} = \vec{y} - \hat{y} = \begin{bmatrix} 2 \\ 3 \end{bmatrix} - \begin{bmatrix} -1/2 \\ 5/2 \end{bmatrix} = \begin{bmatrix} 5/2 \\ 1/2 \end{bmatrix} \]

\[ \vec{y} = \begin{bmatrix} 2 \\ 3 \end{bmatrix} = \begin{bmatrix} -1/2 \\ 5/2 \end{bmatrix} + \begin{bmatrix} 5/2 \\ 1/2 \end{bmatrix} \]

\( \hat{y} \)

\( \vec{z} \)

PAGE 6

notice \( \|\vec{z}\| \) is the shortest distance from \( (2, 3) \) to the line through \( (0, 0) \) and \( (1, -5) \)

If \( \{ \vec{u}_1, \dots, \vec{u}_p \} \) is an orthogonal set and \( \|\vec{u}_i\| = 1 \) for all \( i \), then the set is an orthonormal set

orthogonal \( \rightarrow \) lin. indp.
lin. indp. \( \rightarrow \) orthogonal?No,

lin indp. but not orthogonal