Lecture 29 - More on Orthonormality

Let's look at an example. Consider:

L = {. . ., e^{- 2 i t}, e^{- i t}, e^{0}, e^{i t}, e^{2 i t}, . . .}

We claim that this is an orthonormal list of vectors in the space of continuous complex-valued functions with domain $R$ , equipped with the inner product:

⟨ f, g ⟩ = \frac{1}{2 π} \int_{0}^{2 π} f (t) \overset{―}{g (t)} d t

Note that this is an inner product, which we could verify with the 5 properties of an inner product. Notice that it's interesting that this list is infinite, while being orthonormal. Notice then that:

Each pair of vectors are orthogonal to each other
The norm of any vector in the list is 1.
Recall that:

e^{i θ} = \cos (θ) + i \sin (θ) \Rightarrow \overset{―}{e^{i θ}} = \cos (θ) - i \sin (θ) = \cos (- θ) + i \sin (- θ) = e^{- i θ}

We prove both of these.

\begin{proof}
We first show that any $u, v \in L$ . Notice that then $u = e^{m i t}$ and $v = e^{n i t}$ for some $m, n \in Z$ . Notice if $m \neq n$ then $u \neq v$ so we expect $⟨ u, v ⟩ = 0$ :

⟨ u, v ⟩ = \frac{1}{2 π} \int_{0}^{2 π} e^{m i t} \overset{―}{e^{n i t}} d t = \frac{1}{2 π} \int_{0}^{2 π} e^{m i t} e^{- n i t} d t = \frac{1}{2 π} \int_{0}^{2 π} e^{(m - n) i t} d t

Notice that $m \neq n$ so then $m - n \neq 0$ so then:

⟨ u, v ⟩ = \frac{1}{2 π} \frac{e^{(m - n) i t}}{(m - n) i} |_{0}^{2 π} = 0

by plugging in. If instead $m = n$ then $m - n = 0$ so then:

⟨ u, v ⟩ = \frac{1}{2 π} \int_{0}^{2 π} 1 d t = \frac{2 π}{2 π} = 1

as we expected.
\end{proof}

Graham-Schmidt Process

Suppose $e_{1}, . . ., e_{n}$ is an orthonormal basis for inner product space $V$ . Let $v \in V$ . Thus:

v = α_{1} e_{1} + \dots + α_{n} e_{n}

for all $α_{i} \in F$ . What if we wanted to know that all the $α$ 's are? Normally we'd row reduce, but when you deal with an orthonormal basis, you can calculate these directly! We'll make a formula for them.

Take the inner product of both sides of the original equation with one $e_{i}$ (it doesn't matter), then:

\begin{aligned} ⟨ v, e_{i} ⟩ & = ⟨ α_{1} e_{1} + \dots + α_{n} e_{n}, e_{i} ⟩ \\ = α_{1} ⟨ e_{1}, e_{i} ⟩ + \dots + α_{n} ⟨ e_{n}, e_{i} ⟩ \\ = α_{i} ⟨ e_{i}, e_{i} ⟩ & (all other prods are orthogonal) \\ = α_{i} \end{aligned}

So in any inner product space have some orthonormal basis (ONB) $e_{1}, . . ., e_{n}$ , any vector $V$ can be written uniquely as:

v = ⟨ v, e_{1} ⟩ e_{1} + \dots + ⟨ v, e_{n} ⟩ e_{n}

Suppose further that I have $U = span (e_{1}, . . ., e_{i})$ . Since:

v = \underset{\in U}{\underset{⏟}{⟨ v, e_{1} ⟩ e_{1} + \dots + ⟨ v, e_{i} ⟩ e_{i}}} + \underset{orthogonal to all u \in U}{\underset{⏟}{⟨ v, e_{i + 1} ⟩ e_{i + 1} + \dots + ⟨ v, e_{n} ⟩ e_{n}}} $ $ I t^{'} s e a s y t o s h o w t h e f i r s t u n d e r b r a c e . F o r t h e s e c o n d o n e, n o t i c e t h a t a l l t h e $ ⟨ v, e_{i + 1} ⟩ $ a n d o n w a r d s n e e d t o b e 0, h e n c e s h o w i n g t h i s p r o p e r t y . B u t $ U $ h e r e i s a n i n t e r e s t i n g w a y t o o r t h o g a n l i z e a s p a c e! > [! d e f i n i t i o n] P r o j e c t i o n o f $ v $ o n t o $ u $ . > T o p r o j e c t s o m e $ v \in V $ o n t o $ u \in U $ :>

P_u(v) = \langle v, e_1 \rangle e_1 + \dots + \langle v, e_i \rangle e_i

Gram Schmidt Process

The goal is to start with a basis for $V$ , namely $v_{1}, . . ., v_{n}$ , and find a basis $e_{1}, . . ., e_{n}$ that's orthonormal, such that $span (v_{1}, . . ., v_{i}) = span (e_{1}, . . ., e_{i})$ (we want to build the vectors up one at a time).

Gram Schmidt Algorithm

Given $v_{1}, . . ., v_{n}$ . We want $span (v_{1}) = span (e_{1})$ , so we need $e_{1}$ to be a scalar multiple of $v_{1}$ , while being unit length. Hence:

$e_{1} = \frac{v_{1}}{‖ v_{1} ‖}$ (note $‖ v_{1} ‖ \neq 0$ since $v_{1}$ is in a basis list for $V$ ).

Then suppose we already have $e_{1}, . . ., e_{i - 1}$ already with $e_{1}, . . ., e_{i - 1}$ are orthonormal and $span (e_{1}, . . ., e_{i - 1}) = span (v_{1}, . . ., v_{i - 1})$ . We want:

Where $U = span (e_{1}, . . ., e_{i - 1})$ . Then the next vector we want is:

e_{i} = \frac{v_{i} - P_{u} (v_{i})}{‖ v_{i} - P_{u} (v_{i}) ‖}

Note

You can just normalize all of your vectors at the very end, rather than during the process. You can just get non-normal length $e_{i}^{'}$ 's, then normalize all once at the very end.

Notice that $‖ v_{i} - P_{u} (v) ‖ \neq 0$ since if it was equal then $v_{i} \in span (v_{1}, . . ., v_{i - 1})$ which contradicts it composing a basis of the other vectors. Clearly $e_{i}$ then is unit length.

Further, $e_{i}$ is orthogonal to $e_{1}, . . ., e_{i - 1}$ .

Lastly, $span (e_{1}, . . ., e_{i}) = span (v_{1}, . . ., v_{i})$ since we knew the previous span of $i - 1$ is true, and we get $v_{i}$ spanned from the $e_{i}$ equation to show the new list spans the $v_{1}, . . ., v_{i}$ list.

You continue this process up till the list terminates. Then you have your orthonormal basis $e_{1}, . . ., e_{n}$ .

Consequences of Gram Schmidt

Notice that all that we require is that $v_{1}, . . ., v_{n}$ is our basis, or namely $dim (V) = n$ . Hence:

Every finite dimensional product space has an orthonormal has an orthonormal basis.
Suppose $T \in L (V)$ , and there is a basis for $V$ via $v_{1}, . . ., v_{n}$ such that $M (T, β_{v})$ is UT. Then there exists an orthonormal basis $e_{1}, . . ., e_{n}$ for which $M (T, β_{e})$ is UT.
Every operator $T \in L (V)$ where $dim (V)$ is a complex inner product space has an UT matrix representation with respect to an orthonormal basis (combining the two above)

An Example

Take $v_{1} = (1, 1)$ and $v_{2} = (3, 1)$ . We know these aren't colinear vectors, so clearly they span $R^{2}$ . So $v_{1}, v_{2}$ is a valid basis.

To use Gram Schmidt, scale $v_{1}$ to a unit size to $e_{1} = (2^{- 1 / 2}, 2^{- 1 / 2})$ . Then take $v_{2}$ and subtract the projection of it onto $u = e_{1}$ . Namely:

e_{2} = \frac{(3, 1) - ⟨ v, e_{1} ⟩ e_{1}}{‖ . . . ‖} = [\begin{matrix} \frac{\sqrt{2}}{2} \\ - \frac{\sqrt{2}}{2} \end{matrix}]