Lecture 31 - Last Lecture!

We'll do some cool things with vector projections today!

Consider:

We'll define vector projection in an abstract sense. Recall for our purposes:

Lemma

Suppose $U$ is a finite-dimensional subspace of $V$ . Then:
$V = U \oplus U^{⊥}$
as well as:
$(U^{⊥})^{⊥} = U$

We now show a cool definition:

orthogonal projection

Suppose $U$ is a finite-dimensional subspace of $V$ , the orthogonal projection of $V$ onto $U$ is the operator $P_{u}$ , defined by:

P_{U} (\vec{v}) = u

whenever $v = u + w$ where $u \in U$ and $w \in U^{⊥}$ .

Properties of orthogonal projection

$P_{U} \in L (V)$ ;
$P_{U} (u) = u$ ;
$P_{U} (w) = 0$ for $w \in U^{⊥}$ ;
$range (P_{U}) = U$ ;
$null (P_{U}) = U^{⊥}$ ;
$v - P_{U} (v) \in U^{⊥}$ ;
$(P_{U})^{2} = P_{U}$ ;
$‖ P_{U} (v) ‖ \leq ‖ v ‖$ ;
Given any orthonormal basis for the space $U$ , say $e_{1}, . . ., e_{m}$ , then $P_{U} (v) = \sum_{i = 1}^{m} ⟨ v, e_{i} ⟩ e_{i}$ (this is the $v - P_{U} (v))$ idea coming from Gram-Schmidt.

We'll prove some of these. Let's prove $‖ P_{U} (v) ‖ \leq ‖ v ‖$ :

\begin{proof}
Suppose $v = u + w$ where $u \in U$ and $w \in U^{⊥}$ . Then:

\begin{aligned} ‖ P_{U} (v) ‖^{2} & = ‖ u ‖^{2} \\ \leq ‖ u ‖^{2} + ‖ w ‖^{2} \\ = ‖ u + w ‖^{2} \\ = ‖ v ‖ \end{aligned}

Thus taking both sides reveals the theorem.
\end{proof}

Minimization Problems

We sometimes want to find the minimum norm between two vectors. For some setup, suppose $U$ is a finite-dimensional subspace of $V$ . Then:

‖ v - P_{U} (v) ‖ \leq ‖ v - u ‖

for all $u \in U$ . Namely, the distance from the "plane" $U$ , is less than just taking the distance fr

\begin{proof}

\begin{aligned} ‖ v - P_{U} (v) ‖^{2} & \leq \underset{\in U^{⊥}}{\underset{⏟}{‖ v - P_{U} (v) ‖^{2}}} + \underset{\in U}{\underset{⏟}{‖ P_{U} (v) - u ‖^{2}}} \\ = ‖ v - u ‖^{2} \end{aligned}

Which comes from using the Pythagorean Theorem along with the fact that the two vectors are orthogonal.
\end{proof}
So the best vector to use as an approximation of $v$ , using only vectors in $U$ , would be $P_{u} (v)$ .

An Example

See HW 7 - Inner Product Spaces#^af860a problem. The list is orthonormal with respect to:

⟨ f, g ⟩ = \int_{- π}^{π} f (x) g (x) d x

Let $U_{n} = span (B_{n})$ , where $B_{n}$ was up to the $\cos (n x)$ and $\sin (n x)$ terms from our list. $U_{n}$ is a finite-dimensional subspace of all continuous real valued functions on $- π$ to $π$ .

Note that then we can compute $P_{U_{1}} (e^{x})$ :

U_{1} = span (\frac{1}{\sqrt{2 π}}, \frac{\cos (x)}{\sqrt{π}}, \frac{\sin (x)}{\sqrt{π}})

So then:

P_{U_{1}} (e^{x}) = ⟨ e^{x}, \frac{1}{\sqrt{2 π}} ⟩ \frac{1}{\sqrt{2 π}} + \dots