Chapter 8
By the end of this section, you should understand the definition of inner product space, and be aware of many new examples.
Every vector space in this section is real.
🤔
The familiar dot product, $\bfu\pd\bfv$, of the two vectors $\bfu,\bfv\in\R^n$ is given by $$ \bfu\pd\bfv=u_1v_1+u_2v_2+\cdots + u_nv_n, $$ where $\bfu=(u_1,u_2,\ldots, u_n)$ and $\bfv=(v_1,v_2, \ldots , v_n)$.
This dot product has the following key properties:
This dot product has the following key properties:
Inspired by the dot product on $\R^n$, we define a so-called inner product on a general real vector space by elevating the key properties of the dot product to axioms.
Inspired by the dot product on $\R^n$, we define a so-called inner product on a general real vector space by elevating the key properties of the dot product to axioms.
Accordingly, an inner product on $V$ is a function that takes each ordered pair $(\bfu,\bfv)$ of elements of $V$ to a real number, denoted by $\langle\bfu,\bfv\rangle$, such that for all $\bfu,\bfv,\bfw\in V$ and all $k\in\R$:
| (I1) | $\langle\bfu,\bfv\rangle=\langle\bfv,\bfu\rangle$ |
| (I2) | $\langle\bfu+\bfv,\bfw\rangle=\langle\bfu,\bfw\rangle+\langle\bfv,\bfw\rangle$ |
| (I3) | $\langle k\pd\bfu,\bfv\rangle=k\langle\bfu,\bfv\rangle$ |
| (I4) | $\langle\bfu,\bfu\rangle\geq0$ |
| (I5) | $\langle\bfu,\bfu\rangle=0$ iff $\bfu={\bf 0}$ (where ${\bf 0}$ is the unique zero vector) |
| (I1) | $\langle\bfu,\bfv\rangle=\langle\bfv,\bfu\rangle$ |
| (I2) | $\langle\bfu+\bfv,\bfw\rangle=\langle\bfu,\bfw\rangle+\langle\bfv,\bfw\rangle$ |
| (I3) | $\langle k\pd\bfu,\bfv\rangle=k\langle\bfu,\bfv\rangle$ |
| (I4) | $\langle\bfu,\bfu\rangle\geq0$ |
| (I5) | $\langle\bfu,\bfu\rangle=0$ iff $\bfu={\bf 0}$ (where ${\bf 0}$ is the unique zero vector) |
A vector space with an inner product associated to it is called an inner product space. As we are assuming that all vector spaces are real in this section, we have thus introduced the notion of a real inner product space. One can also define inner products on complex vector spaces, thereby introducing complex inner product spaces, but they are beyond the scope of these lectures.
Let $\u, \v\in \R^n$ be given. That is \[ \u = \left(u_1, u_2, \cdots, u_n\right)\quad \text{and} \quad \v = \left(v_1, v_2, \cdots, v_n\right), \] and let $w_1,w_2,\ldots,w_n\gt 0$ be scalars, then we define the inner product by \[ \langle \u, \v \rangle = w_1u_1v_1 + w_2u_2v_2 + \cdots + w_nu_nv_n , \] known as the Weighted dot product; the scalars $w_1,w_2,\ldots,w_n$ are called weights.
Note: This inner product $\langle , \rangle$ with $w_k=1$, for all $k$, is nothing but the usual dot product.
Let $\u, \v\in \R^n$, and let $A\in M_{n,n}(\R)$ be invertible matrix. Then \[ \langle \u, \v\rangle = \left(A\u\right)\pd \left(A\v\right) \] defines an inner product.
Note 1: $\left(A\u\right)\pd \left(A\v\right)$ $=\left(A\u\right)^{T} A\v $ $=\u^{T} A^{T} A\v .$
Let $\u, \v\in \R^n$, and let $A\in M_{n,n}(\R)$ be invertible matrix. Then \[ \langle \u, \v\rangle = \left(A\u\right)\pd \left(A\v\right) \] defines an inner product.
Note 1: $\left(A\u\right)\pd \left(A\v\right)$ $=\left(A\u\right)^{T} A\v $ $=\u^{T} A^{T} A\v .$
Note 2: The weighted dot product of the previous example corresponds to \[ A = \begin{pmatrix} \sqrt{w_1} & 0 & 0 & \cdots & 0\\ 0 & \sqrt{w_2} & 0 & \cdots & 0 \\ 0 & 0 & \sqrt{w_3} & \cdots & \vdots\\ \vdots & \vdots & \vdots & \ddots & 0\\ 0 & 0 & \cdots & 0 & \sqrt{w_n}\\ \end{pmatrix} \]
Let $\u, \v\in M_{n,n}(\R)$. That is \[ \u = \begin{pmatrix} a_{11} & \cdots & a_{1n}\\ \vdots & \ddots & \vdots\\ a_{n1} & \cdots & a_{nn}\\ \end{pmatrix} \quad \text{and} \quad \v = \begin{pmatrix} b_{11} & \cdots & b_{1n}\\ \vdots & \ddots & \vdots\\ b_{n1} & \cdots & b_{nn}\\ \end{pmatrix}. \]
Then $\ds \langle \u , \v \rangle = \mbox{Tr}\left(\v^{T} \u\right)$ is an inner product, where the trace "$\mbox{Tr}$" is defined as \[ \mbox{Tr} \begin{pmatrix} a_{11} & \cdots & a_{1n}\\ \vdots & \ddots & \vdots\\ a_{n1} & \cdots & a_{nn}\\ \end{pmatrix} = a_{11}+a_{22}+\cdots + a_{nn}. \]
Let $\u, \v\in M_{n,n}(\R)$. That is \[ \u = \begin{pmatrix} a_{11} & \cdots & a_{1n}\\ \vdots & \ddots & \vdots\\ a_{n1} & \cdots & a_{nn}\\ \end{pmatrix} \quad \text{and} \quad \v = \begin{pmatrix} b_{11} & \cdots & b_{1n}\\ \vdots & \ddots & \vdots\\ b_{n1} & \cdots & b_{nn}\\ \end{pmatrix}. \]
Then $\ds \langle \u , \v \rangle = \mbox{Tr}\left(\v^{T} \u\right)$ is an inner product, where the trace "$\mbox{Tr}$" is defined as \[ \mbox{Tr} \begin{pmatrix} a_{11} & \cdots & a_{1n}\\ \vdots & \ddots & \vdots\\ a_{n1} & \cdots & a_{nn}\\ \end{pmatrix} = a_{11}+a_{22}+\cdots + a_{nn}. \]
Note: Since $\,\mbox{Tr} \left(A^T\right) = \mbox{Tr}(A)$ $\,\Ra \mbox{Tr} \left(\v^T \u\right) = \mbox{Tr}(\u^T\v).$
For $\mathbf p, \mathbf q \in P_n\left(\R\right)$, we write \[ \begin{eqnarray} \mathbf p \equiv p(x) & = & a_0 + a_1 x+ \cdots + a_n x^n, \\ \mathbf q \equiv q(x) & = & b_0 + b_1 x+ \cdots + b_n x^n.\\ \end{eqnarray} \]
Then \[ \langle \mathbf p , \mathbf q \rangle = a_0b_0 + a_1b_1 + \cdots +a_nb_n \] defines an inner product called standard inner product on $P_n\left(\R\right).$
Let $x_0,x_1,\ldots, x_n\in \R$ be distinct and $\mathbf p, \mathbf q\in P_n(\R)$ be as in Example 8.6
Then \[ \langle \mathbf p , \mathbf q \rangle = p\left(x_0\right)q\left(x_0\right)+ p\left(x_1\right)q\left(x_1\right) + \cdots + p\left(x_n\right)q\left(x_n\right) \] defines an inner product on $P_n\left(\R\right)$, called evaluation inner product.
Let $\mathbf f, \mathbf g\in C[a,b]$. Then
$\ds \langle \mathbf f, \mathbf g \rangle = \int_a^b f(x)g(x)~dx\;$ defines an inner product.
Let $\mathbf f, \mathbf g\in C[a,b]$. Then $\ds \langle \mathbf f, \mathbf g \rangle = \int_a^b f(x)g(x)~dx\;$ defines an inner product.
Proof: We can easily check (I1)-(I3) by properties of the integral. (Recall Definition 8.2)
(I4): We have that $\ds \langle \mathbf f, \mathbf f\rangle = \int_a^b f^2(x)~dx .$ Since $\left(\,f(x)\right)^2\geq 0$ and $f$ is continuous on $[a,b]$ $\;\Ra \ds\int_a^b f^2(x)~dx\geq 0$ $\;\Ra \langle \mathbf f, \mathbf f\rangle \geq 0.$
(I5): Here we need that $\ds \int_a^b f^2(x)~dx = 0 $ which in turn requires $\left(\,f(x)\right)^2= 0$ $\;\Ra \ds f(x) = 0$ on $[a,b],$ that is, $\;\Ra \mathbf f = \mathbf 0.$ $\quad \blacksquare$