← Back to Math 010

6.2 Angle and Orthogonality in Inner Product Spaces: Class Notes

Definition

The angle $\theta$ between vectors $𝐮$ and $𝐯$ in a real inner product space $V$ is ${\displaystyle \theta ={\cos }^{-1}\left(\frac{⟨𝐮,𝐯⟩}{\Vert 𝐮\Vert \Vert 𝐯\Vert }\right)}$.

This is the same formula we saw with the dot product, where now $\Vert 𝐮\Vert =\sqrt{⟨𝐮,𝐮⟩}$ comes from the inner product.

Example

Find the cosine of the angle between $A$ and $B$ with respect to the standard inner product on $M_{22}$.

$$A=\left[\begin{array}{cc}2& -4\\ -1& 3\end{array}\right],B=\left[\begin{array}{cc}-3& 1\\ 4& 2\end{array}\right]$$

Solution

The standard inner product on $M_{22}$ multiplies entries in matching positions and adds the products:

$$⟨A,B⟩=2(-3)+(-4)(1)+(-1)(4)+3(2)=-6-4-4+6=-8$$

$$\begin{array}{cc}\hfill {\displaystyle \Vert A\Vert }& {\displaystyle =\sqrt{⟨A,A⟩}=\sqrt{4+16+1+9}=\sqrt{30}}\hfill \\ \hfill {\displaystyle \Vert B\Vert }& {\displaystyle =\sqrt{⟨B,B⟩}=\sqrt{9+1+16+4}=\sqrt{30}}\hfill \end{array}$$

$${\displaystyle \cos \theta =\frac{⟨A,B⟩}{\Vert A\Vert \Vert B\Vert }=\frac{-8}{\sqrt{30}\sqrt{30}}=-\frac{8}{30}=-\frac{4}{15}}$$

(So $\theta \approx {105.5}^{\circ }$.)

Theorem 6.2.1 Cauchy-Schwarz Inequality (generalization of Theorem 3.2.4)

This is a necessary result in order to compute the angle between vectors in an inner product space: it guarantees that the quotient inside the inverse cosine is between $-1$ and 1.

Proof

If $𝐮=\mathrm{𝟎}$, then $⟨𝐮,𝐯⟩=0$ and $\Vert 𝐮\Vert =0$, so both sides are 0 and the inequality holds.

If $𝐮\ne \mathrm{𝟎}$, then let $⟨𝐮,𝐮⟩=a$, $2⟨𝐮,𝐯⟩=b$, $⟨𝐯,𝐯⟩=c$, and let $t$ be any real number. By the positivity axiom,

$$\begin{array}{cc}\hfill {\displaystyle 0}& {\displaystyle \le ⟨t𝐮+𝐯,t𝐮+𝐯⟩=⟨𝐮,𝐮⟩t^2+2⟨𝐮,𝐯⟩t+⟨𝐯,𝐯⟩}\hfill \\ \hfill {\displaystyle 0}& {\displaystyle \le a{t}^2+bt+c}\hfill \end{array}$$

Since $𝐮\ne \mathrm{𝟎}$, $a>0$, so $y=a{t}^2+bt+c$ is a parabola that opens upward and is never below the axis. It either touches the axis once or does not cross it at all, so it has at most one real root. Then its discriminant is not positive:

Two sketches of $y=a{t}^2+bt+c\ge 0$ with $a>0$. Left: the parabola opens upward and touches the horizontal axis at exactly one point (one real root). Right: the parabola opens upward and lies entirely above the axis (no real roots). In both cases $b^2-4ac\le 0$.

$$b^2-4ac\le 0$$

Substituting for $a$, $b$, and $c$:

$$4⟨𝐮,𝐯{⟩}^2-4⟨𝐮,𝐮⟩⟨𝐯,𝐯⟩\le 0$$

Dividing by 4 and moving the second term to the right side gives the inequality in three equivalent forms:

$$⟨𝐮,𝐯{⟩}^2\le ⟨𝐮,𝐮⟩⟨𝐯,𝐯⟩\text{(one form)}$$

$$⟨𝐮,𝐯{⟩}^2\le \Vert 𝐮{\Vert }^2\Vert 𝐯{\Vert }^2\text{(another form)}$$

$$|⟨𝐮,𝐯⟩|\le \Vert 𝐮\Vert \Vert 𝐯\Vert \text{(yet another form)}$$

The last form comes from taking square roots of both sides, and it is the statement of the theorem.

Theorem 6.2.2 Triangle inequalities (generalization of Theorem 3.2.5)

Proof of (a)

$$\begin{array}{cc}\hfill {\displaystyle \Vert 𝐮+𝐯{\Vert }^2}& {\displaystyle =⟨𝐮+𝐯,𝐮+𝐯⟩}\hfill \\ \hfill {\displaystyle }& {\displaystyle =⟨𝐮,𝐮⟩+2⟨𝐮,𝐯⟩+⟨𝐯,𝐯⟩}\hfill \\ \hfill {\displaystyle }& {\displaystyle \le ⟨𝐮,𝐮⟩+2|⟨𝐮,𝐯⟩|+⟨𝐯,𝐯⟩\text{(property of absolute value)}}\hfill \\ \hfill {\displaystyle }& {\displaystyle \le \Vert 𝐮{\Vert }^2+2\Vert 𝐮\Vert \Vert 𝐯\Vert +\Vert 𝐯{\Vert }^2\text{(Cauchy-Schwarz inequality)}}\hfill \\ \hfill {\displaystyle }& {\displaystyle ={(\Vert 𝐮\Vert +\Vert 𝐯\Vert )}^2}\hfill \end{array}$$

The first inequality holds because any real number is at most its absolute value, so $2⟨𝐮,𝐯⟩\le 2|⟨𝐮,𝐯⟩|$. The second uses the Cauchy-Schwarz inequality, $|⟨𝐮,𝐯⟩|\le \Vert 𝐮\Vert \Vert 𝐯\Vert$. Taking square roots (both sides are nonnegative) finishes the proof.

Proof of (b)

Subtract and add $𝐰$ inside the norm, then use part (a) in the form $\Vert 𝐚+𝐛\Vert \le \Vert 𝐚\Vert +\Vert 𝐛\Vert$:

$$\begin{array}{cc}\hfill {\displaystyle d(𝐮,𝐯)}& {\displaystyle =\Vert 𝐮-𝐯\Vert =\Vert (𝐮-𝐰)+(𝐰-𝐯)\Vert }\hfill \\ \hfill {\displaystyle }& {\displaystyle \le \Vert 𝐮-𝐰\Vert +\Vert 𝐰-𝐯\Vert }\hfill \\ \hfill {\displaystyle }& {\displaystyle =d(𝐮,𝐰)+d(𝐰,𝐯)}\hfill \end{array}$$

Definition

Two vectors $𝐮$ and $𝐯$ in a real inner product space $V$ are orthogonal if $⟨𝐮,𝐯⟩=0$.

#10

Show that the vectors are orthogonal with respect to the standard inner product on $P_2$.

$𝐩=2-3x+x^2$, $𝐪=4+2x-2x^2$

Solution

Multiply matching coefficients and add:

$$⟨𝐩,𝐪⟩=2(4)+(-3)(2)+1(-2)=8-6-2=0$$

#33

Let $C[-1,1]$ have the integral inner product ${\displaystyle ⟨𝐩,𝐪⟩={\int }_{-1}^{1}p(x)q(x)dx}$ and let $𝐩=p(x)=x^2-x$ and $𝐪=q(x)=x+1$.

a. Find $⟨𝐩,𝐪⟩$.

b. Find $\Vert 𝐩\Vert$ and $\Vert 𝐪\Vert$.

Solution

a. Expand: $(x^2-x)(x+1)=x^3+x^2-x^2-x=x^3-x$, an odd function, so its integral over $[-1,1]$ is 0.

$${\displaystyle ⟨𝐩,𝐪⟩={\int }_{-1}^1(x^2-x)(x+1)dx={\int }_{-1}^1(x^3-x)dx=0}$$

b. In $(x^2-x{)}^2=x^4-2x^3+x^2$, the term $-2x^3$ is odd, so its integral over $[-1,1]$ is $0$ and it drops out.

$${\displaystyle \begin{array}{cc}\hfill {\displaystyle \Vert 𝐩\Vert }& {\displaystyle ={({\int }_{-1}^1(x^2-x{)}^{2}dx)}^{1/2}={({\int }_{-1}^1(x^4-2x^3+x^2)dx)}^{1/2}}\hfill \\ \hfill {\displaystyle }& {\displaystyle ={({\int }_{-1}^1(x^4+x^2)dx)}^{1/2}={(\frac{2}{5}+\frac{2}{3})}^{1/2}=\sqrt{\frac{16}{15}}=\frac{4}{\sqrt{15}}}\hfill \end{array}}$$

Similarly, in $(x+1{)}^2=x^2+2x+1$ the odd term $2x$ integrates to $0$:

$${\displaystyle \begin{array}{cc}\hfill {\displaystyle \Vert 𝐪\Vert }& {\displaystyle ={({\int }_{-1}^1(x+1{)}^{2}dx)}^{1/2}={({\int }_{-1}^1(x^2+2x+1)dx)}^{1/2}}\hfill \\ \hfill {\displaystyle }& {\displaystyle ={(\frac{2}{3}+2)}^{1/2}=\sqrt{\frac{8}{3}}=\frac{2\sqrt{2}}{\sqrt{3}}}\hfill \end{array}}$$

Explore

What is $\Vert 𝐩+𝐪{\Vert }^2$? What about $\Vert 𝐩{\Vert }^2+\Vert 𝐪{\Vert }^2$?

Using $𝐩$ and $𝐪$ from #33: $𝐩+𝐪=(x^2-x)+(x+1)=x^2+1$, so

$${\displaystyle \Vert 𝐩+𝐪{\Vert }^2={\int }_{-1}^1(x^2+1{)}^{2}dx={\int }_{-1}^1(x^4+2x^2+1)dx=\frac{2}{5}+\frac{4}{3}+2=\frac{56}{15}}$$

$${\displaystyle \Vert 𝐩{\Vert }^2+\Vert 𝐪{\Vert }^2=\frac{16}{15}+\frac{8}{3}·\frac{5}{5}=\frac{16}{15}+\frac{40}{15}=\frac{56}{15}}$$

So $\Vert 𝐩{\Vert }^2+\Vert 𝐪{\Vert }^2=\Vert 𝐩+𝐪{\Vert }^2$. This works because $𝐩$ and $𝐪$ are orthogonal (#33a).

Theorem 6.2.3 Generalized Theorem of Pythagoras

Proof

$$\begin{array}{cc}\hfill {\displaystyle \Vert 𝐮+𝐯{\Vert }^2}& {\displaystyle =⟨𝐮+𝐯,𝐮+𝐯⟩=\Vert 𝐮{\Vert }^2+2⟨𝐮,𝐯⟩+\Vert 𝐯{\Vert }^2}\hfill \\ \hfill {\displaystyle }& {\displaystyle =\Vert 𝐮{\Vert }^2+\Vert 𝐯{\Vert }^2}\hfill \end{array}$$

since $𝐮⟂𝐯$ means $⟨𝐮,𝐯⟩=0$.

Example

Let $𝐩=p(x)=1-2x^2$, $𝐪=q(x)=4-2x+x^2$, $𝐫=r(x)=x+2x^2$ and let $P_2$ have the standard inner product.

a. Compute the following: $⟨𝐩,𝐪⟩$ and $⟨𝐪,𝐫⟩$.

b. Now let $P_2$ have the inner product $⟨𝐩,𝐪⟩=a_0{b}_0+a_1{b}_1+k{a}_2{b}_2$. Find $k$ so that $𝐩$ and $𝐪$ are orthogonal.

c. Using this new inner product, compute $⟨𝐪,𝐫⟩$.

Solution

a. The polynomial $𝐩$ has no $x$ term and $𝐫$ has no constant term.

$$\begin{array}{cc}\hfill {\displaystyle ⟨𝐩,𝐪⟩}& {\displaystyle =1(4)+0(-2)+(-2)(1)=2}\hfill \\ \hfill {\displaystyle ⟨𝐪,𝐫⟩}& {\displaystyle =4(0)+(-2)(1)+1(2)=0}\hfill \end{array}$$

b. With the new inner product, set $⟨𝐩,𝐪⟩$ equal to 0 and solve for $k$:

$$⟨𝐩,𝐪⟩=1(4)+0(-2)+k(-2)(1)=0\Rightarrow 4-2k=0\Rightarrow k=2$$

c. Now $k=2$, so the $x^2$ coefficients are multiplied by 2:

$$⟨𝐪,𝐫⟩=4(0)+(-2)(1)+2(1)(2)=0-2+4=2$$

Under the standard inner product $𝐪$ and $𝐫$ were orthogonal, but under the new one they are not. Orthogonality is based on the vectors and how the inner product is defined.

#17

Do there exist scalars $k$ and $l$ such that the vectors ${𝐩}_1=2+kx+6x^2$, ${𝐩}_2=l+5x+3x^2$, ${𝐩}_3=1+2x+3x^2$ are mutually orthogonal with respect to the standard inner product on $P_2$?

Solution

Each pair must have inner product 0:

$$\begin{array}{cc}\hfill {\displaystyle {𝐩}_1⟂{𝐩}_2:}& {\displaystyle 2l+5k+18=0}\hfill \\ \hfill {\displaystyle {𝐩}_2⟂{𝐩}_3:}& {\displaystyle l+10+9=0\Rightarrow l=-19}\hfill \\ \hfill {\displaystyle {𝐩}_1⟂{𝐩}_3:}& {\displaystyle 2+2k+18=0\Rightarrow k=-10}\hfill \end{array}$$

The second and third conditions force $l=-19$ and $k=-10$. But then the first condition fails:

$$2(-19)+5(-10)+18=-38-50+18=-70\ne 0$$

Definition

(inner product space analog of Definition 2 in Section 4.9) If $W$ is a subspace of a real inner product space $V$, then the set of all vectors in $V$ that are orthogonal to every vector in $W$ is called the orthogonal complement of $W$ and is denoted by the symbol $W^{⟂}$ (pronounced “$W$ perp”).

Theorem 6.2.4 (generalization of Theorem 4.9.6 (a) and (b))

Theorem 6.2.5 (generalization of Theorem 4.9.6(c))

#28

Find a basis for the orthogonal complement of the subspace of $R^5$ spanned by the vectors.

${𝐯}_1=(1,4,5,6,9)$, ${𝐯}_2=(3,-2,1,4,-1)$, ${𝐯}_3=(-1,0,-1,-2,-1)$, ${𝐯}_4=(2,3,5,7,8)$

Solution

Let $𝐰$ be a vector in the orthogonal complement. Then ${𝐯}_1·𝐰=0$, and so on for ${𝐯}_2$, ${𝐯}_3$, and ${𝐯}_4$. For example, the first condition is

$$1w_1+4w_2+5w_3+6w_4+9w_5=0$$

So put the vectors in the rows of a matrix and row reduce:

$$\left[\begin{array}{ccccc}1& 4& 5& 6& 9\\ 3& -2& 1& 4& -1\\ -1& 0& -1& -2& -1\\ 2& 3& 5& 7& 8\end{array}\right]⟶\left[\begin{array}{ccccc}1& 0& 1& 2& 1\\ 0& 1& 1& 1& 2\\ 0& 0& 0& 0& 0\\ 0& 0& 0& 0& 0\end{array}\right]$$

Writing the unknowns as $x_1,x_2,\dots ,x_5$, the nonzero rows give

$$\begin{array}{cc}\hfill {\displaystyle x_1}& {\displaystyle =-x_3-2x_4-x_5}\hfill \\ \hfill {\displaystyle x_2}& {\displaystyle =-x_3-x_4-2x_5}\hfill \end{array}$$

Let $r=x_3$, $s=x_4$, and $t=x_5$ (the free variables). Then

$$𝐱=\left[\begin{array}{c}-r-2s-t\\ -r-s-2t\\ r\\ s\\ t\end{array}\right]=r\left[\begin{array}{c}-1\\ -1\\ 1\\ 0\\ 0\end{array}\right]+s\left[\begin{array}{c}-2\\ -1\\ 0\\ 1\\ 0\end{array}\right]+t\left[\begin{array}{c}-1\\ -2\\ 0\\ 0\\ 1\end{array}\right]$$

These three vectors form a basis for the orthogonal complement. Multiplying each one by $-1$ gives another basis, with fewer minus signs: