Mathematical Analysis
Lecture 23
12.1 The derivative in $\mathbb R^n$
Motivation
Recall that for $f \colon I \subseteq \R \to \R,$ we defined the derivative at $c$ as
$\ds\lim_{x \to c} \frac{f(x)-f(c)}{x-c} $ $\ds= \lim_{h\to 0}\frac{f(c+h)-f(c)}{h}.$
12.1 The derivative in $\mathbb R^n$
Motivation
So, in general, we have that for $f \colon U\subseteq \R \to \R$ the derivative at any $x_0\in U$ is
$$\ds \lim_{h\to 0}\frac{f(x_0+h)-f(x_0)}{h}= f'(x_0).$$
12.1 The derivative in $\mathbb R^n$
Motivation
$$\ds \lim_{h\to 0}\frac{f(x_0+h)-f(x_0)}{h}= f'(x_0).$$
12.1 The derivative in $\mathbb R^n$
Motivation
To generalize this, we interpret $f'(x_0)$ as a linear map acting on the vector $h$.
12.1 The derivative in $\mathbb R^n$
Motivation
Then we can say that $f'(x_0)$ is the unique linear map from $\R$ to $\R$ such that the mapping $g:U\to \R$ given by
$u \mapsto g(u) = f(x_0) + f'(x_0)(u-x_0)$
is tangent to $f$ at $x_0.$
12.1 The derivative in $\mathbb R^n$
Motivation
Thus $f'(x_0)\in L\left(\R^1,\R^1\right),$ which is the best linear approximation of how $f$ changes near $x_0$.
Can we extend this to $L\left(\R^n, \R^m\right)$? 🤔
12.1 The derivative in $\mathbb R^n$
Motivation
In general, we have that for $f \colon U\subseteq \R \to \R$ the derivative at any $x\in U$ is
$$\ds \lim_{h\to 0}\frac{f(x+h)-f(x)}{h}= f'(x).$$
$$\ds \lim_{h\to 0}\frac{f(x+h)-f(x)}{h} = A.$$
$$\ds \iff \lim_{h\to 0}\abs{\frac{f(x+h)-f(x)}{h} - A }= 0$$
$$\ds \iff \lim_{h\to 0}\abs{\frac{f(x+h)-f(x)-Ah}{h} }= 0$$
$$\ds \iff \lim_{h\to 0}\frac{\abs{f(x+h)-f(x)-Ah}}{\abs{h}} = 0.$$
In this context, $A\in L\left(\R^1,\R^1\right).$
12.1 The derivative in $\mathbb R^n$
Motivation
$$\ds \iff \lim_{h\to 0}\frac{\abs{f(x+h)-f(x)-Ah}}{\abs{h}} = 0.$$
In this context, $A\in L\left(\R^1, \R^1\right).$
That is, $A$ is a linear map in one dimension.
Now we can extend this to vector valued
mappings of a
vector variable. 😃
12.1 The derivative in $\mathbb R^n$
Definition 12.1.1. Let $U \subseteq \R^n$ be open and $f \colon U \to \R^m$ a function. We say $f$ is differentiable at $x \in U$ if there exists an $A \in L(\R^n,\R^m)$ such that
$\ds \lim_{\substack{h \to 0\\h\in \R^n}} \frac{\snorm{f(x+h)-f(x) - Ah}}{\snorm{h}} = 0 . $
12.1 The derivative in $\mathbb R^n$
$\ds \lim_{\substack{h \to 0\\h\in \R^n}} \frac{\snorm{f(x+h)-f(x) - Ah}}{\snorm{h}} = 0 . $
We write $Df(x) := A,$ or $f'(x) := A,$ and we say $A$ is the derivative of $f$ at $x$. When $f$ is differentiable at every $x \in U,$ we say simply that $f$ is differentiable.
12.1 The derivative in $\mathbb R^n$
$ \ds\lim_{\substack{h \to 0\\h\in \R^n}} \frac{\snorm{f(x+h)-f(x) - Ah}}{\snorm{h}} = 0 . $
Remarks:
- Under this definition, the derivative $Df(x)$ of $f$ at $x$ is not a number but rather a linear mapping. For each $A\in L\left(\R^n,\R^m\right)$ there is a matrix $\mathcal M(A)\in M_{m \times n}.$
- It is also common to set $y := x + h\in \R^n$ and rewrite the limit as $$ \lim_{\substack{y \to x}} \frac{\snorm{f(y)-f(x) - A(y-x)}}{\snorm{y-x}} = 0 .$$
12.1 The derivative in $\mathbb R^n$
Remarks:
- We can extend the definition of derivative to other kind functions. For instance, consider the mapping $X \mapsto \det(X).$ For every $X\in L(\R^n)$ we compute the determinant $\det(X)\in \R.$ In this case we have \[ \lim_{\substack{H \to 0\\H\in L\left(\R^n\right)}} \frac{\snorm{\det(X+H)-\det(X) - AH}}{\snorm{H}} = 0 . \] Here $A: \text{GL}(\R^n)\to \R\,$ is a linear operator, where $\text{GL}\left(\R^n\right)$ is the set of invertible linear operators.
12.1 The derivative in $\mathbb R^n$
Theorem 12.1.1. Let $U \subseteq \R^n$ be an open subset and $f \colon U \to \R^m$ a function. If $x \in U$ and there exist $A,B \in L(\R^n,\R^m)$ such that
$\ds \lim_{h \to 0} \frac{\snorm{f(x+h)-f(x) - Ah}}{\snorm{h}} = 0$
and $\;\;\ds \lim_{h \to 0} \frac{\snorm{f(x+h)-f(x) - Bh}}{\snorm{h}} = 0 , \quad\;\;\;$
then $A=B.$
Example 12.1.1.
If $f(x) = Ax$ for a linear mapping $A,$ then $f'(x) = A$.
To prove this note that
$\dfrac{\snorm{f(x+h)-f(x) - Ah}}{\snorm{h}} $ $ = \dfrac{\snorm{A(x+h)-Ax - Ah}}{\snorm{h}} $
$\quad \quad \;\,= \dfrac{0}{\snorm{h}}$ $ = 0 . $
$\Ra \ds \lim_{\substack{h \to 0\\h\in \R^n}} \frac{\snorm{f(x+h)-f(x) - Ah}}{\snorm{h}} = 0. $
Hence $f'(x) = A.$
Example 12.1.2.
Let $f \colon \R^2 \to \R^2$ be defined by \begin{equation*} f(x,y) \coloneqq (1+x+2y+x^2,2x+3y+xy). \end{equation*}
Claim: $f$ is differentiable at the origin.
Proof. If the derivative exists, it is in $L\left(\R^2,\R^2\right),$ so it can be represented by a $2$-by-$2$ matrix $$A = \left[\begin{matrix}a&b\\c&d\end{matrix}\right].$$
Example 12.1.2.
Let $f \colon \R^2 \to \R^2$ be defined by $ f(x,y) \coloneqq (1+x+2y+x^2,2x+3y+xy). $
Claim: $f$ is differentiable at the origin.
Proof. For $h = (h_1,h_2),$ using the definition of derivative we have that
$\ds\frac{\snorm{f(x+h)-f(x) - Ah}}{\snorm{h}}$
$\ds= \frac{\snorm{f\big((0,0)+(h_1,h_2)\big)-f(0,0) - A(h_1,h_2)}}{\snorm{(h_1,h_2)}}$
$\ds =\frac{\snorm{ f(h_1,h_2)-f(0,0) - (ah_1 +bh_2 , ch_1+dh_2)} }{\snorm{(h_1,h_2)}} $
Example 12.1.2.
Let $f \colon \R^2 \to \R^2$ be defined by $ f(x,y) \coloneqq (1+x+2y+x^2,2x+3y+xy). $
Claim: $f$ is differentiable at the origin.
Proof. For $h = (h_1,h_2),$ using the definition of derivative we have that
$\ds\frac{\snorm{f(x+h)-f(x) - Ah}}{\snorm{h}}$
$\ds=\frac{\snorm{ f(h_1,h_2)-f(0,0) - (ah_1 +bh_2 , ch_1+dh_2)} }{\snorm{(h_1,h_2)}}$
$\ds =\frac{\sqrt{ {\bigl((1-a)h_1 + (2-b)h_2 + h_1^2\bigr)}^2 + {\bigl((2-c)h_1 + (3-d)h_2 + h_1h_2\bigr)}^2}}{\sqrt{h_1^2+h_2^2}} . $
Example 12.1.2.
Let $f \colon \R^2 \to \R^2$ be defined by $ f(x,y) \coloneqq (1+x+2y+x^2,2x+3y+xy). $
Claim: $f$ is differentiable at the origin.
Proof. For $h = (h_1,h_2),$ using the definition of derivative we have that
$\ds\frac{\snorm{f(x+h)-f(x) - Ah}}{\snorm{h}}$
$\ds =\frac{\sqrt{ {\bigl((1-a)h_1 + (2-b)h_2 + h_1^2\bigr)}^2 + {\bigl((2-c)h_1 + (3-d)h_2 + h_1h_2\bigr)}^2}}{\sqrt{h_1^2+h_2^2}} . $
which is a very long and complicated expression... 😬
Example 12.1.2.
Let $f \colon \R^2 \to \R^2$ be defined by $ f(x,y) \coloneqq (1+x+2y+x^2,2x+3y+xy). $
Claim: $f$ is differentiable at the origin.
Proof.
$\ds \frac{\sqrt{ {\bigl((1-a)h_1 + (2-b)h_2 + h_1^2\bigr)}^2 + {\bigl((2-c)h_1 + (3-d)h_2 + h_1h_2\bigr)}^2}}{\sqrt{h_1^2+h_2^2}} . $
😃 If we choose $a=1,$ $b=2,$ $c=2,$ $d=3;$ then
$\ds\frac{\sqrt{ h_1^4 + h_1^2h_2^2}}{\sqrt{h_1^2+h_2^2}}$ $\ds= \sabs{h_1} \frac{\sqrt{ h_1^2 + h_2^2}}{\sqrt{h_1^2+h_2^2}}$ $\ds= \sabs{h_1} .$
Example 12.1.2.
Let $f \colon \R^2 \to \R^2$ be defined by $ f(x,y) \coloneqq (1+x+2y+x^2,2x+3y+xy). $
Claim: $f$ is differentiable at the origin.
Proof. This implies that
$\ds\frac{\snorm{f(x+h)-f(x) - Ah}}{\snorm{h}} = \abs{h_1}.$
This expression goes to $0$ as $h\to 0.$
Therefore $f$ is differentiable at the origin. Furthermore, the derivative $f'(0)$ is represented by the matrix \[ \left[\begin{matrix}1&2\\2&3\end{matrix}\right]. \]
12.1.1 Properties of the derivative
Theorem 12.1.2. Let $U \subseteq \R^n$ be open and $f \colon U \to \R^m$ be differentiable at $p \in U.$ Then $f$ is continuous at $p$.
What is the definition of continuity for $f \colon \R^n \to \R^m$?
12.1.1 Properties of the derivative
Theorem 12.1.2. Let $U \subset \R^n$ be open and $f \colon U \to \R^m$ be differentiable at $p \in U.$ Then $f$ is continuous at $p$.
What is the definition of continuity for $f \colon \R^n \to \R^m$?
$f$ is continuous at $p\in\R^n$ if $\ds \forall \epsilon > 0, \exists \delta > 0 $
such that $ ||x - p || \lt \delta $ $\;\Rightarrow \;|| f(x) - f( p) || \lt \epsilon.$
i.e., $\ds \lim_{x\to p} f(x) = f(p) \;$ or $\ds \;f(x)\to f(p) \text{ as } x\to p. $
If $h:=x-p,\;$ then $\ds \; f(p+h)\to f(p) \text{ as } h\to 0. $
12.1.1 Properties of the derivative
Theorem 12.1.2. Let $U \subset \R^n$ be open and $f \colon U \to \R^m$ be differentiable at $p \in U.$ Then $f$ is continuous at $p$.
Proof. Since $f$ is differentiable at $p,$ then
$\ds \lim_{\substack{h \to 0\\h\in \R^n}} \frac{\snorm{f(p+h)-f(p) - f'(p)h}}{\snorm{h}} = 0.$
This means that the mapping $h\mapsto f'(p) h$ is
a linear mapping between finite-dimensional spaces.
Then $f'(p)$ is continuous
(Why?)
and
$f'(p) h \to 0$ as $h \to 0$ $\Ra$ $f(p+h) \to f(p)$ as $h \to 0$.
Therefore, $f$ is continuous at $p.$ $\; \bs$
12.1.1 Properties of the derivative
Theorem 12.1.3. Suppose $U \subseteq \R^n$ is open, $f \colon U \to \R^m$ and $g \colon U \to \R^m$ are differentiable at $p,$ and $\alpha \in \R$. Then the functions $f+g$ and $\alpha f$ are differentiable at $p$ and and
1. $ (f+g)'(p) = f'(p) + g'(p) ,$
2. $ \; (\alpha f)'(p) = \alpha f'(p). \qquad \quad \, $
Proof of Theorem 12.1.3
Let $h \in \R^n,$ with $h \not= 0$. For first item, consider
$\ds \frac{\norm{f(p+h)+g(p+h)-\bigl(f(p)+g(p)\bigr) - \bigl(f'(p) + g'(p)\bigr)h}}{\snorm{h}}$
$\ds \leq \frac{\norm{f(p+h)-f(p) - f'(p)h}}{\snorm{h}} + \frac{\norm{g(p+h)-g(p) - g'(p)h}}{\snorm{h}} , $
which tends to $0$ as $h\to 0$ since $f$ and $g$ are differentiable. Hence $(f+g)'(p) = f'(p) + g'(p).$
Proof of Theorem 12.1.3
To prove the second item, consider the following
$\ds \frac{\norm{\alpha f(p+h) - \alpha f(p) - \alpha f'(p)h}}{\snorm{h}} \qquad\qquad \qquad $
$ = \ds \sabs{\alpha} \frac{\norm{f(p+h))-f(p) - f'(p)h}}{\snorm{h}} . $
which tends to $0$ as $h\to 0$ since $f$ is differentiable. Therefore $ (\alpha f)'(p) = \alpha f'(p) .\; \bs$
12.1.1 Properties of the derivative
Theorem 12.1.4. (Chain rule) Let $U \subseteq \R^n$ be open and let $f \colon U \to \R^m$ be differentiable at $p \in U.$ Let $V \subset \R^m$ be open, $f(U) \subset V$ and let $g \colon V \to \R^\ell$ be differentiable at $f(p).$ Then \begin{equation*} F(x) = g\bigl(f(x)\bigr) \end{equation*} is differentiable at $p$ and \begin{equation*} F'(p) = g'\bigl(f(p)\bigr) f'(p) . \end{equation*}