← Back to Math 010

6.1 Inner Products: Class Notes

Definition

An inner product on a real vector space $V$ is a function that associates a real number $⟨𝐮,𝐯⟩$ with each pair of vectors in $V$ in such a way that the following axioms are satisfied for all vectors $𝐮$, $𝐯$, and $𝐰$ in $V$ and all scalars $k$.

1. $⟨𝐮,𝐯⟩=⟨𝐯,𝐮⟩$ (symmetry axiom)
2. $⟨𝐮+𝐯,𝐰⟩=⟨𝐮,𝐰⟩+⟨𝐯,𝐰⟩$ (additivity axiom)
3. $⟨k𝐮,𝐯⟩=k⟨𝐮,𝐯⟩$ (homogeneity axiom)
4. $⟨𝐯,𝐯⟩\ge 0$ and $⟨𝐯,𝐯⟩=0$ if and only if $𝐯=\mathrm{𝟎}$ (positivity axiom)

Definition

A real vector space with an inner product is called a real inner product space.

Note that an inner product is a category of operation that is performed on vectors in a vector space. As long as an operation satisfies these four axioms, it is an inner product.

Building up the structures:

• A collection of objects is a set.
• A set together with a particularly defined addition and scalar multiplication is a vector space.
• A vector space with an inner product is an inner product space.

Examples of Inner Products

• The dot product $⟨𝐮,𝐯⟩=𝐮·𝐯=u_1{v}_1+u_2{v}_2+\dots +u_n{v}_n$ is the Euclidean inner product or standard inner product on $R^n$. [$R^n$ with the Euclidean inner product is called Euclidean $𝐧$-space.]
• If $𝐮=(u_1,u_2,\dots ,u_n)$ and $𝐯=(v_1,v_2,\dots ,v_n)$ are vectors in $R^n$ and $w_1,w_2,\dots ,w_n$ are positive real numbers, then the formula $⟨𝐮,𝐯⟩=w_1{u}_1{v}_1+w_2{u}_2{v}_2+\dots +w_n{u}_n{v}_n$ is called the weighted Euclidean inner product with weights ${\mathbf{w}}_{\mathbf{1}},{\mathbf{w}}_{\mathbf{2}},\dots ,{\mathbf{w}}_{\mathbf{n}}$.
• On $M_{nn}$, the set of $n\times n$ matrices: If $𝐮=U$ and $𝐯=V$ are matrices in the vector space $M_{nn}$, then the formula $⟨𝐮,𝐯⟩=\mathrm{tr}(U^{T}V)$ is the standard inner product on $M_{nn}$.
• If $𝐩=a_0+a_{1}x+\dots +a_n{x}^n$ and $𝐪=b_0+b_{1}x+\dots +b_n{x}^n$ are polynomials in $P_n$, then the standard inner product on $P_n$ is $⟨𝐩,𝐪⟩=a_0{b}_0+a_1{b}_1+\dots +a_n{b}_n$. (Note the similarity in form to the dot product.)
• If $𝐩=a_0+a_{1}x+\dots +a_n{x}^n$ and $𝐪=b_0+b_{1}x+\dots +b_n{x}^n$ are polynomials in $P_n$ and if $x_0,x_1,\dots ,x_n$ are distinct real numbers, then the formula $⟨𝐩,𝐪⟩=p(x_0)q(x_0)+p(x_1)q(x_1)$ $\dots +p(x_n)q(x_n)$ is the evaluation inner product at $x_0,x_1,\dots ,x_n$.
• If $𝐟=f(x)$ and $𝐠=g(x)$ are two functions in $C[a,b]$, then ${\displaystyle ⟨𝐟,𝐠⟩={\int }_a^{b}f(x)g(x)dx}$ defines an inner product on $C[a,b]$.
• If $𝐮$ and $𝐯$ are vectors in $R^n$ expressed in column form, $A$ is an invertible $n\times n$ matrix, and $𝐮·𝐯$ is the Euclidean inner product on $R^n$, then the formula $⟨𝐮,𝐯⟩=A𝐮·A𝐯$ is the inner product on ${𝐑}^{𝐧}$ generated by $𝐀$. This is an example of a matrix inner product.

Tip: For the evaluation inner product, it helps to make a table with columns for $x$, $p(x)$, and $q(x)$ (one row for each point $x_0,x_1,\dots ,x_n$), then multiply across each row and add the products (see #16).

#10

Compute the standard inner product on $M_{22}$ of the given matrices.

$$U=\left[\begin{array}{cc}1& 2\\ -3& 5\end{array}\right],V=\left[\begin{array}{cc}4& 6\\ 0& 8\end{array}\right]$$

Solution

The standard inner product on $M_{22}$ is $⟨𝐮,𝐯⟩=\mathrm{tr}(U^{T}V)$.

$$\begin{array}{cc}\hfill {\displaystyle U^{T}V}& {\displaystyle =\left[\begin{array}{cc}1& -3\\ 2& 5\end{array}\right]\left[\begin{array}{cc}4& 6\\ 0& 8\end{array}\right]=\left[\begin{array}{cc}4& -18\\ 8& 52\end{array}\right]}\hfill \\ \hfill {\displaystyle ⟨𝐮,𝐯⟩}& {\displaystyle =\mathrm{tr}(U^{T}V)=4+52=56}\hfill \end{array}$$

Note that the order doesn’t matter (symmetry axiom):

$$⟨𝐯,𝐮⟩=\mathrm{tr}(V^{T}U)=\mathrm{tr}\left(\left[\begin{array}{cc}4& 0\\ 6& 8\end{array}\right]\left[\begin{array}{cc}1& 2\\ -3& 5\end{array}\right]\right)=\mathrm{tr}\left[\begin{array}{cc}4& 8\\ -18& 52\end{array}\right]=56$$

Why the trace works like a dot product. Let

$$A=\left[\begin{array}{cc}{a}_1& a_2\\ a_3& a_4\end{array}\right],B=\left[\begin{array}{cc}{b}_1& b_2\\ b_3& b_4\end{array}\right]$$

$$A^{T}B=\left[\begin{array}{cc}{a}_1& a_3\\ a_2& a_4\end{array}\right]\left[\begin{array}{cc}{b}_1& b_2\\ b_3& b_4\end{array}\right]=\left[\begin{array}{cc}{a}_1{b}_1+a_3{b}_3& a_1{b}_2+a_3{b}_4\\ a_2{b}_1+a_4{b}_3& a_2{b}_2+a_4{b}_4\end{array}\right]$$

The trace adds only the diagonal entries:

$$\mathrm{tr}(A^{T}B)=a_1{b}_1+a_2{b}_2+a_3{b}_3+a_4{b}_4$$

So $\mathrm{tr}(A^{T}B)$ multiplies matching entries of $A$ and $B$ and adds the products: it is basically a dot product of the entries. For #10, pair each entry of $U$ with the entry in the same position of $V$:

$$⟨𝐮,𝐯⟩=1(4)+2(6)+(-3)(0)+5(8)=4+12+0+40=56$$

#11

Find the standard inner product on $P_2$ of the given polynomials.

$𝐩=-2+x+3x^2$, $𝐪=4-7x^2$

Solution

For $𝐩=a_0+a_{1}x+a_2{x}^2$ and $𝐪=b_0+b_{1}x+b_2{x}^2$, $⟨𝐩,𝐪⟩=a_0{b}_0+a_1{b}_1+a_2{b}_2$. The polynomial $𝐪$ has no $x$ term, so $b_1=0$.

$$⟨𝐩,𝐪⟩=-2(4)+1(0)+3(-7)=-8+0-21=-29$$

#16

A sequence of sample points is given. Use the evaluation inner product on $P_3$ at those sample points to find $⟨𝐩,𝐪⟩$ for the polynomials $𝐩=x+x^3$ and $𝐪=1+x^2$.

$x_0=-1$, $x_1=0$, $x_2=1$, $x_3=2$

Solution

$$⟨𝐩,𝐪⟩=p(x_0)q(x_0)+p(x_1)q(x_1)+p(x_2)q(x_2)+p(x_3)q(x_3)$$

Evaluate both polynomials at each sample point, then multiply the two values at each point:

Add the products:

$$⟨𝐩,𝐪⟩=-2(2)+0(1)+2(2)+10(5)=-4+0+4+50=50$$

#8

Use the inner product on $R^2$ generated by the matrix $A$ to find $⟨𝐮,𝐯⟩$ for the vectors $𝐮=(0,-3)$ and $𝐯=(6,2)$.

$$A=\left[\begin{array}{cc}2& 1\\ -1& 3\end{array}\right]$$

Solution

The inner product generated by $A$ is $⟨𝐮,𝐯⟩=A𝐮·A𝐯$. Write the vectors as columns and multiply:

$$A𝐮=\left[\begin{array}{cc}2& 1\\ -1& 3\end{array}\right]\left[\begin{array}{c}0\\ -3\end{array}\right]=\left[\begin{array}{c}-3\\ -9\end{array}\right],A𝐯=\left[\begin{array}{cc}2& 1\\ -1& 3\end{array}\right]\left[\begin{array}{c}6\\ 2\end{array}\right]=\left[\begin{array}{c}14\\ 0\end{array}\right]$$

$$⟨𝐮,𝐯⟩=(-3,-9)·(14,0)=-3(14)+(-9)(0)=-42$$

In general, since $𝐮·𝐯={𝐯}^T𝐮$ (see Table 1 in Section 3.2), we have

$$⟨𝐮,𝐯⟩=A𝐮·A𝐯=(A𝐯{)}^{T}A𝐮={𝐯}^T{A}^{T}A𝐮$$

Example: A weighted Euclidean inner product

For this class, homework is worth 10%, exams 60%, and the final 30% of the grade. Suppose we have 4 exams, each with 100 points possible. The final has 100 points possible, and the total possible homework points is 320.

$𝐮=(265,350,85)$ is the points earned, and ${\displaystyle 𝐯=(\frac{1}{320},\frac{1}{400},\frac{1}{100})}$ is the base for each category.

The Euclidean inner product $⟨𝐮,𝐯⟩=𝐮·𝐯$ does not factor in the category weights. Instead, define

$$⟨𝐮,𝐯⟩=10u_1{v}_1+60u_2{v}_2+30u_3{v}_3$$

Then

$${\displaystyle ⟨𝐮,𝐯⟩=10\left(\frac{265}{320}\right)+60\left(\frac{350}{400}\right)+30\left(\frac{85}{100}\right)\approx 86}$$

so Student A has about 86% in the class. This is a weighted Euclidean inner product with weights $w_1=10$, $w_2=60$, and $w_3=30$.

Considering a matrix inner product, a weighted Euclidean inner product is generated by

$$A=\left[\begin{array}{cccc}\sqrt{w_1}& 0& \cdots & 0\\ 0& \sqrt{w_2}& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & \sqrt{w_n}\end{array}\right]\text{so that}{A}^{T}A=\left[\begin{array}{cccc}{w}_1& 0& \cdots & 0\\ 0& w_2& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & w_n\end{array}\right]$$

Then ${𝐯}^T{A}^{T}A𝐮=w_1{u}_1{v}_1+w_2{u}_2{v}_2+\dots +w_n{u}_n{v}_n$.

Definition

If $V$ is a real inner product space, then the norm or length of a vector $𝐯$ in $V$ is denoted by $\Vert 𝐯\Vert$ and is defined by $\Vert 𝐯\Vert =\sqrt{⟨𝐯,𝐯⟩}$ and the distance between two vectors is denoted by $d(𝐮,𝐯)$ and is defined by $d(𝐮,𝐯)=\Vert 𝐮-𝐯\Vert =\sqrt{⟨𝐮-𝐯,𝐮-𝐯⟩}$. A vector of norm 1 is called a unit vector.

#1

Let $R^2$ have the weighted Euclidean inner product $⟨𝐮,𝐯⟩=2u_1{v}_1+3u_2{v}_2$ and let $𝐮=(1,1)$, $𝐯=(3,2)$, $𝐰=(0,-1)$, and $k=3$. Compute the stated quantities.

a. $⟨𝐮,𝐯⟩$  b. $⟨k𝐮,𝐯⟩$  c. $⟨𝐮+𝐯,𝐰⟩$  d. $\Vert 𝐯\Vert$  e. $d(𝐮,𝐯)$  f. $\Vert 𝐮-k𝐯\Vert$

Solution

a. $⟨𝐮,𝐯⟩=2(1)(3)+3(1)(2)=6+6=12$

b. Here $k𝐮=3(1,1)=(3,3)$, so

$$⟨k𝐮,𝐯⟩=2(3)(3)+3(3)(2)=18+18=36$$

Note: $⟨3𝐮,𝐯⟩=3⟨𝐮,𝐯⟩=3(12)=36$, as the homogeneity axiom says.

c. Here $𝐮+𝐯=(1+3,1+2)=(4,3)$, so

$$⟨𝐮+𝐯,𝐰⟩=2(4)(0)+3(3)(-1)=0-9=-9$$

Note: $⟨𝐮,𝐰⟩=2(1)(0)+3(1)(-1)=-3$ and $⟨𝐯,𝐰⟩=2(3)(0)+3(2)(-1)=-6$, and $-3+(-6)=-9$, as the additivity axiom says.

d.

$$\Vert 𝐯\Vert =\sqrt{⟨𝐯,𝐯⟩}=\sqrt{2(3)(3)+3(2)(2)}=\sqrt{18+12}=\sqrt{30}$$

e. $d(𝐮,𝐯)=\sqrt{⟨𝐮-𝐯,𝐮-𝐯⟩}$, where $𝐮-𝐯=(1,1)-(3,2)=(-2,-1)$.

$$d(𝐮,𝐯)=\sqrt{2(-2{)}^2+3(-1{)}^2}=\sqrt{8+3}=\sqrt{11}$$

f. Here $𝐮-k𝐯=(1,1)-3(3,2)$ $=(1,1)-(9,6)=(-8,-5)$, so

$$\Vert 𝐮-k𝐯\Vert =\sqrt{⟨𝐮-k𝐯,𝐮-k𝐯⟩}=\sqrt{2(-8{)}^2+3(-5{)}^2}=\sqrt{128+75}=\sqrt{203}$$

Theorem 6.1.1 Norms and Distances in Inner Product Spaces

#37

Let the vector space $P_2$ have the inner product ${\displaystyle ⟨𝐩,𝐪⟩={\int }_{-1}^{1}p(x)q(x)dx}$. Find the following for $𝐩=1$ and $𝐪=x^2$.

a. $⟨𝐩,𝐪⟩$  b. $d(𝐩,𝐪)$  c. $\Vert 𝐩\Vert$  d. $\Vert 𝐪\Vert$

Solution

a.

$${\displaystyle ⟨𝐩,𝐪⟩={\int }_{-1}^{1}1·x^{2}dx={\frac{1}{3}{x}^3|}_{-1}^1=\frac{1}{3}-(-\frac{1}{3})=\frac{2}{3}}$$

b.

$${\displaystyle \begin{array}{cc}\hfill {\displaystyle d(𝐩,𝐪)}& {\displaystyle =\sqrt{⟨𝐩-𝐪,𝐩-𝐪⟩}={({\int }_{-1}^1(1-x^2{)}^{2}dx)}^{1/2}}\hfill \\ \hfill {\displaystyle }& {\displaystyle ={({\int }_{-1}^1(1-2x^2+x^4)dx)}^{1/2}}\hfill \end{array}}$$

The integrand is even, so the integral from $-1$ to 1 is twice the integral from 0 to 1:

$${\displaystyle \begin{array}{cc}\hfill {\displaystyle d(𝐩,𝐪)}& {\displaystyle ={(2{\int }_0^1(1-2x^2+x^4)dx)}^{1/2}={\left(2{(x-\frac{2}{3}{x}^3+\frac{1}{5}{x}^5)|}_0^1\right)}^{1/2}}\hfill \\ \hfill {\displaystyle }& {\displaystyle ={\left(2(1-\frac{2}{3}+\frac{1}{5})\right)}^{1/2}=\sqrt{\frac{16}{15}}=\frac{4}{\sqrt{15}}}\hfill \end{array}}$$

c.

$${\displaystyle \Vert 𝐩\Vert =\sqrt{⟨𝐩,𝐩⟩}={\left({\int }_{-1}^{1}1dx\right)}^{1/2}=\sqrt{2}}$$

d. Here $𝐪=x^2$, so $q(x)q(x)=x^4$.

$${\displaystyle \Vert 𝐪\Vert ={\left({\int }_{-1}^1{x}^{4}dx\right)}^{1/2}={\left({\frac{1}{5}{x}^5|}_{-1}^1\right)}^{1/2}=\sqrt{\frac{2}{5}}}$$

Definition

If $V$ is an inner product space, then the set of points in $V$ that satisfy $\Vert 𝐮\Vert =1$ is called the unit sphere in $V$ (or the unit circle in the case where $V=R^2$).

The Euclidean inner product in $R^2$ gives us the unit circle. A weighted Euclidean inner product in $R^2$ gives us a distorted circle, also known as an ellipse.

In the case of the integral inner product, like in #37, the unit sphere consists of all functions $f\in C[a,b]$ such that

$${\displaystyle {\int }_a^b[f(x){]}^{2}dx=1}$$

Examples:

$${\displaystyle {\int }_{-1}^1\frac{1}{2}dx=1\Rightarrow f(x)=\frac{1}{\sqrt{2}}\text{is on the unit sphere in }C[-1,1]}$$

$${\displaystyle {\int }_0^{\pi /2}\sin xdx=1\Rightarrow f(x)=\sqrt{\sin x}\text{is on the unit sphere in }C[0,\pi /2]}$$

Theorem 6.1.2 Algebraic Properties of Inner Products (generalization of Theorem 3.2.3)

Proving these is a good practice exercise.