Download original notes
PAGE 1

6.7 Inner Product Spaces

For \( \vec{u}, \vec{v} \) in \( \mathbb{R}^n \), the standard inner product (dot product) is

\[ \langle \vec{u}, \vec{v} \rangle = u_1 v_1 + u_2 v_2 + \dots + u_n v_n \]

4 properties of inner product

  1. \( \langle \vec{u}, \vec{v} \rangle = \langle \vec{v}, \vec{u} \rangle \)
  2. \( \langle \vec{u} + \vec{v}, \vec{w} \rangle = \langle \vec{u}, \vec{w} \rangle + \langle \vec{v}, \vec{w} \rangle \)
  3. \( \langle c\vec{u}, \vec{v} \rangle = c \langle \vec{u}, \vec{v} \rangle \)
  4. \( \langle \vec{u}, \vec{u} \rangle \geq 0 \) and \( \langle \vec{u}, \vec{u} \rangle = 0 \) if and only if \( \vec{u} = \vec{0} \)

Any vector space with the inner product according to the 4 properties above is an inner product space.

BUT we can define inner product in other ways.

e.g. \( \langle \vec{u}, \vec{v} \rangle = 3u_1 v_1 + 4u_2 v_2 \) in \( \mathbb{R}^2 \)

this satisfies the 4 axioms above

PAGE 2

If inner product changes, then "length" and "orthogonality" also change their means.

for example, if \( \langle \vec{u}, \vec{v} \rangle = 3u_1 v_1 + 4u_2 v_2 \)

\[ \vec{x} = \begin{bmatrix} 3 \\ 1 \end{bmatrix} \quad \vec{y} = \begin{bmatrix} 5 \\ -1 \end{bmatrix} \]
\[ \| \vec{x} \| = \sqrt{\langle \vec{x}, \vec{x} \rangle} = \sqrt{(3)(3)(3) + (4)(1)(1)} = \sqrt{31} \]

\( \sqrt{31} \) is the "length" of \( \vec{x} \) in this inner product space

Find \( \vec{z} \) such that \( \vec{z} \) is "orthogonal" to \( \vec{y} \)?

\( \vec{z} = \begin{bmatrix} z_1 \\ z_2 \end{bmatrix} \) want \( \langle \vec{z}, \vec{y} \rangle = 0 \)

\[ \langle \vec{z}, \vec{y} \rangle = 3(z_1)(5) + 4(z_2)(-1) = 0 \]
\[ z_1 = \frac{4}{15} z_2 \]
\[ \vec{z} = \begin{bmatrix} \frac{4}{15} z_2 \\ z_2 \end{bmatrix} = z_2 \begin{bmatrix} 4/15 \\ 1 \end{bmatrix} \]

any vector parallel to \( \begin{bmatrix} 4 \\ 15 \end{bmatrix} \) is \( \perp \) to \( \vec{y} \)

PAGE 3

Distance, length, and orthogonality of polynomials?

Let polynomials be defined as:

\[ p(t) = t \quad \text{and} \quad q(t) = t^2 \]
A coordinate graph showing a linear function p(t)=t and a parabolic function q(t)=t^2 intersecting at the origin.

What does \( \langle p, q \rangle \) mean?

We need to define the inner product. One way is to base it on values of \( p \) and \( q \) at several \( t \), for example, \( t = -1, 0, 1 \).

Then define:

\[ \langle p, q \rangle = p(-1)q(-1) + p(0)q(0) + p(1)q(1) \]

Calculating the norm squared of \( p \):

\[ \begin{aligned} \langle p, p \rangle = \|p\|^2 &= p(-1)p(-1) + p(0)p(0) + p(1)p(1) \\ &= (-1)(-1) + (0)(0) + (1)(1) = 2 \end{aligned} \]

So "length" of \( p(t) \) is \( \|p\| = \sqrt{2} \)

PAGE 4

Calculating the norm squared of \( q \):

\[ \begin{aligned} \langle q, q \rangle = \|q\|^2 &= q(-1)q(-1) + q(0)q(0) + q(1)q(1) \\ &= 1 + 0 + 1 = 2 \end{aligned} \]

Thus, \( \|q\| = \sqrt{2} \).

Calculating the inner product of \( p \) and \( q \):

\[ \begin{aligned} \langle p, q \rangle &= p(-1)q(-1) + p(0)q(0) + p(1)q(1) \\ &= (-1)(1) + (0)(0) + (1)(1) = 0 \end{aligned} \]

So \( p(t) = t \) and \( q(t) = t^2 \) are orthogonal.

Linear Independence vs. Orthogonality

The set \( \{1, t, t^2\} \) is linearly independent, but is it orthogonal?

Note: \( t \) and \( t^2 \) are orthogonal (from work above).

Let \( p_1(t) = 1, p_2(t) = t, p_3(t) = t^2 \).

\[ \begin{aligned} \langle p_1, p_2 \rangle &= p_1(-1)p_2(-1) + p_1(0)p_2(0) + p_1(1)p_2(1) \\ &= (1)(-1) + (1)(0) + (1)(1) = 0 \end{aligned} \]

So \( p_1 \perp p_2 \).

\[ \begin{aligned} \langle p_1, p_3 \rangle &= p_1(-1)p_3(-1) + p_1(0)p_3(0) + p_1(1)p_3(1) \\ &= (1)(1) + (1)(0) + (1)(1) = 2 \end{aligned} \]

Since \( \langle p_1, p_3 \rangle \neq 0 \), the set is not orthogonal.

PAGE 5

Gram-Schmidt Process for Polynomials

Page 5

\( \{1, t, t^2\} \) is not orthogonal, but we can apply Gram-Schmidt to find an orthogonal set.

Initial Basis:

\( p_1(t) = 1, \quad p_2(t) = t, \quad p_3(t) = t^2 \)

Inner Product Definition:

\( \langle p, q \rangle = p(-1)q(-1) + p(0)q(0) + p(1)q(1) \)

Step-by-Step Orthogonalization

1. First basis element:

\( q_1(t) = p_1(t) = 1 \)

2. Second basis element:

\( q_2(t) = p_2(t) = t \) because we know from last page, \( p_1 \perp p_2 \)

3. Third basis element:

\[ q_3(t) = p_3(t) - \frac{\langle p_3, q_1 \rangle}{\langle q_1, q_1 \rangle} q_1 - \frac{\langle p_3, q_2 \rangle}{\langle q_2, q_2 \rangle} q_2 \] \[ = t^2 - \frac{2}{3}(1) - \frac{0}{2}t = t^2 - \frac{2}{3} = 3t^2 - 2 \]

So \( \{1, t, 3t^2 - 2\} \) is an orthogonal set and an orthogonal basis for \( \mathbb{P}_2 \)

PAGE 6

Best Approximation in \( \mathbb{P}_2 \)

Page 6

What is the best approximation of \( t^3 \) in \( \mathbb{P}_2 \)?

\( \rightarrow \) the "shadow" of \( t^3 \) in \( \mathbb{P}_2 \)

A geometric sketch showing a vector projecting a shadow onto a plane representing the subspace P2.

Let \( r(t) = t^3 \), find \( \hat{r}(t) = \text{proj}_{\mathbb{P}_2} t^3 \)

Using orthogonal basis: \( \{1, t, 3t^2 - 2\} \)

\( q_1=1, q_2=t, q_3=3t^2-2 \)

\[ \hat{r}(t) = \frac{\langle r, q_1 \rangle}{\langle q_1, q_1 \rangle} q_1 + \frac{\langle r, q_2 \rangle}{\langle q_2, q_2 \rangle} q_2 + \frac{\langle r, q_3 \rangle}{\langle q_3, q_3 \rangle} q_3 \] \[ = \dots = t \]

(because \( t \) and \( t^3 \) are closest at \( t = -1, 0, 1 \))

Why these?

Why not sample ALL \( t \) between \( t=a, t=b \)?

Graph showing curves of t, t squared, and t cubed intersecting at points -1, 0, and 1 on the axes.

Standard Inner Product

So, we can do "standard" inner product:

\[ \langle f, g \rangle = \int_{a}^{b} f(t)g(t) \, dt \]

for \( f, g \) in \( C[a, b] \) (all continuous functions on \( a \le t \le b \))

PAGE 7

Example: Orthogonality of Trigonometric Functions

Consider the functions:

\[ f(t) = \cos t, \quad g(t) = \sin t, \quad -\pi \le t \le \pi \]

The inner product is calculated as:

\[ \langle f, g \rangle = \int_{-\pi}^{\pi} \cos t \sin t \, dt = \left. \frac{1}{2} \sin^2 t \right|_{-\pi}^{\pi} = 0 \]

So \( \sin t \) and \( \cos t \) are orthogonal.

So are \( \cos(nt) \) and \( \cos(mt) \) when \( n \neq m \) (and \( \sin(nt) \) and \( \sin(mt) \)).

Best Approximation

Using the inner product definition \( \langle f, g \rangle = \int_{a}^{b} f(t)g(t) \, dt \), the best approximation of \( t^3 \) by a function in \( \mathbb{P}_2 \) on \( -1 \le t \le 1 \) is:

\[ \frac{3}{5}t \]