\documentclass{article}
\title{Analysis I}
\author{Prof. T.~W.~K\"orner}
\date{Lent 2003}
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{pifont}
\usepackage{cancel}
\usepackage{hyperref}
\usepackage{graphicx}
\hypersetup{colorlinks=true, linkcolor=red}

\theoremstyle{definition} 
\newtheorem{AYTheorem}{Theorem}[section]
\newtheorem{AYLemma}[AYTheorem]{Lemma}
\newtheorem{AYCorollary}[AYTheorem]{Corollary}
\newtheorem{AYDefinition}[AYTheorem]{Definition}
\newtheorem{AYExample}[AYTheorem]{Example}
\newtheorem{AYExercise}[AYTheorem]{Exercise}
\newtheorem{AYConvention}[AYTheorem]{Convention}
\newtheorem{AYStatement}[AYTheorem]{Statement}
\newtheorem{AYFact}[AYTheorem]{Fact}
\newtheorem{AYAxiom}[AYTheorem]{Axiom}
\setlength{\parindent}{0pt}

\begin{document}
\maketitle
\tableofcontents
\newpage
\underline{Lecturer's preamble}\\

This course centres around 1 idea, the re-founding of calculus on a rigorous basis.

This is essentially based on the definition of a limit:

\begin{equation*}
\forall \epsilon > 0 \ \exists n_0(\epsilon) \text{ such that } |a_n-a| < \epsilon \ \forall n \geq n_0(\epsilon)
\end{equation*}

From this we get the FUNDAMENTAL THEOREM OF ANALYSIS

\mbox{Every strictly increasing sequence bounded above tends to a limit}

This does not work in $\mathbb{Q}$, e.g. look at the decimal expansion of $\sqrt{2}$, as you add more and more digits it is clearly increasing, it is bounded above by 2, but its limit ($\sqrt{2}$) does not exist in $\mathbb{Q}$. However, it always works on $\mathbb{R}$

Using this axiom and the laws of algebra (e.g. $a + b = b + a$), we can now refound the calculus.\\

NOTE: We can now split proofs and results into two groups, those which are mere algebra (i.e. proof also works on $\mathbb{Q}$) and those using analysis.

\section{Why do we bother?}

It is surprising how many people think that analysis consists in the difficult proofs of obvious theorems. All we need know, they say, is what a limit is, the definition of continuity and the definition of the derivative. All the rest is `intuitively clear'.

If pressed they will agree that these definitions apply as much to the rationals ${\mathbb Q}$ as to the real numbers ${\mathbb R}$. They then have to explain the following interesting example.
\begin{AYExample}
\label{RationalExample}
If $f:{\mathbb Q}\rightarrow{\mathbb Q}$ is given by
\begin{align*}
f(x)&=-1 \qquad \text{if $x^{2}<2$,}\\
f(x)&=1 \qquad \text{otherwise,}
\end{align*}

then

(i) $f$ is a continuous function with $f(0)=-1$, $f(2)=1$ yet there does not exist a $c$ with $f(c)=0$,

(ii) $f$ is a differentiable function with $f'(x)=0$ for all $x$ yet $f$ is not constant.
\end{AYExample}

What is the difference between $\mathbb{R}$ and $\mathbb{Q}$ which makes calculus work on one even though it fails on the other? Both are `ordered fields', that is, both support operations of `addition' and `multiplication' together with a relation `greater than' (`order') with the properties that we expect. If the reader is interested she will find a complete list of the appropriate axioms in texts like the altogether excellent book of Spivak~\cite{Spivak} and its many rather less excellent competitors, but, interesting as
such things may be, they are irrelevant to our purpose which is not to consider the shared properties of $\mathbb{R}$ and $\mathbb{Q}$ but to identify a \emph{difference} between the two systems which will enable us to exclude the possibility of a function like that of Example~\ref{RationalExample} for functions from $\mathbb{R}$ to $\mathbb{R}$.

To state the difference we need only recall a definition
from course~C3.
\begin{AYDefinition}
\label{ConvergenceFirstDefinition}
If $a_n \in \mathbb{R}$ for each $n \geq 1$ and $a \in \mathbb{R}$ then we say that $a_n \rightarrow a$ if given $\epsilon > 0$ we can find an $n_0 (\epsilon)$ such that
\begin{equation*}
|a_{n}-a| < \epsilon \text{ for all $n \geq n_0 (\epsilon)$}
\end{equation*}
\end{AYDefinition}

The key property of the reals, the \emph{fundamental axiom} which makes everything work was also stated in the course~C3.

\begin{AYAxiom}[The fundamental axiom of analysis]
If $a_n \in \mathbb{R}$ for each $n \geq 1$, $A \in \mathbb{R}$ and $a_1 \leq a_2 \leq a_3 \leq \ldots$ and $a_n < A$ for each $n$ then there exists an $a \in \mathbb{R}$ such that $a_n \rightarrow a$ as $n \rightarrow \infty$.
\end{AYAxiom}

Less ponderously, and just as rigorously, the fundamental axiom for the real numbers says \emph{every increasing sequence bounded above tends to a limit}.

Everything which depends on the fundamental axiom is analysis, everything else is mere algebra.

\section{The axiom of Archimedes}

We start by proving the following results on limits, some of which you saw proved in course~C3.
\begin{AYLemma}
\label{SequenceLimitProperties}
(i) The limit is unique. That is, if $a_n \rightarrow a$ and $a_n \rightarrow b$ as $n \rightarrow \infty$ then $a=b$.

(ii) If $a_n \rightarrow a$ as $n \rightarrow \infty$ and $n(1) < n(2) < n(3) \ldots$ then $a_{n(j)} \rightarrow a$ as $j \rightarrow \infty$.

(iii) If $a_n = c$ for all $n$ then $a_n \rightarrow c$ as $n \rightarrow \infty$.

(iv) If $a_n \rightarrow a$ and $b_n \rightarrow b$ as $n \rightarrow \infty$ then $a_n + b_n \rightarrow a + b$.

(v) If $a_n \rightarrow a$ and $b_n \rightarrow b$ as $n \rightarrow \infty$ then $a_n b_n \rightarrow ab$.

(vi) If $a_n \rightarrow a$ as $n \rightarrow \infty$, $a_n \neq 0$ for each $n$ and $a \neq 0$, then $a_n^{-1} \rightarrow  a^{-1}$.

(vii) If $a_n\leq A$ for each $n$ and $a_n\rightarrow a$ as $n\rightarrow\infty$ then $a\leq A$.
\end{AYLemma}

\begin{proof}
i) If $b \neq a$ then $|b-a| > 0$ so set $\epsilon = |\frac{b - a}{4}|$\\
There exists an $n_0$ such that $|a_n - a| < \epsilon \ \forall n > n_0$\\
There exists an $n_1$ such that $|a_n - b| < \epsilon \ \forall n > n_1$\\
Set $n_2 = \max \{n_0, n_1\}$\\
We have $|a_{n_2} - a| < \epsilon$ and $|a_{n_2} - b| < \epsilon$ so $|a-b| < 2 \epsilon = |\frac{a-b}{2}| < |a-b|$\#\\

ii) Suppose $a_n \rightarrow a$\\
Then given $\epsilon > 0 \ \exists n_0$ such that $|a_n - a| < \epsilon \ \forall n \geq n_0$\\
If $j \geq n_0$ then $n(j) \geq n_0$, generates a subsequence, so $|a_{n(j)} - a| < \epsilon$, thus $a_{n(j)} \rightarrow a$\\

iii) If $_n = c$ then given $\epsilon > 0$ set $n_0(\epsilon) = 1$.\\
Then $|a_n - c| = |c-c| = 0 < \epsilon \ \forall n \geq n_0(\epsilon)$ as required\\

iv) We know that\\
Given $\epsilon > 0 \ \exists n_0(\epsilon)$ such that $|a_n - a| < \epsilon \ \forall n \geq n_0(\epsilon)$\\
Given $\epsilon > 0 \ \exists n_1(\epsilon)$ such that $|b_n - b| < \epsilon \ \forall n \geq n_1(\epsilon)$\\

Thus if $n \geq n_2(\epsilon) = \max\{n_0(\frac{\epsilon}{2}), n_1(\frac{\epsilon}{2})\}$

\begin{align*}
|(a_n + b_n) - (a - b)| &= |(a_n - a) + (b_n - b)|\\
&\leq |a_n - a| + |b_n -b|\\
&< \frac{\epsilon}{2} + \frac{\epsilon}{2}
\end{align*}
for $n \geq n_2(\epsilon)$\\

v) As before, we have that \\
There exists an $n_0$ such that $|a_n - a| < \epsilon \ \forall n > n_0$\\
There exists an $n_1$ such that $|a_n - b| < \epsilon \ \forall n > n_1$\\

Thus if $n \geq n_2(\frac{\epsilon}{3}) = \max\{n_1(1), n_0(\frac{\epsilon}{2(|b|+1)}), n_1(\frac{\epsilon}{2(|a|+1)})\}$

\begin{align*}
\text{Then }|a_n b_n - ab| &\leq |a_n b_n - a b_n + a b_n - ab|\\
&= |(a_n - a)b_n + a(b_n - b)|\\
&\leq |(a_n - a)b_n| + |a(b_n - b)|\\
&= |a_n - a||b_n| + |a||b_n - b|\\
&\leq |a_n - a|(|b| + 1) + |a| |b_n - b|\\
&< \frac{\epsilon}{2(|b| + 1)} + \frac{|a| \epsilon}{2(|a| + 1)}\\
&\leq \frac{\epsilon}{2} + \frac{\epsilon}{2} = \epsilon
\end{align*}

Note: If $n \geq n_1(1), |b_n -b| < 1$ so $|b_n| < |b|+1$\\
We use $\frac{\epsilon}{2(|a| + 1)}$ instead of $\frac{\epsilon}{2|a|}$ in case $a =0$\\

vi) Again, given $\epsilon > 0 \ \exists n_0(\epsilon)$ such that $|a_n - a| < \epsilon$\\
Thus if $n \geq n_1(\epsilon) = \max\{n_0(\frac{|a|}{2}), n_0(\frac{\epsilon|a|^2}{2})\}$

\begin{align*}
\Big|\frac{1}{a_n} - \frac{1}{a}\Big| = \Big|\frac{a - a_n}{aa_n}\Big| &= \frac{|a-a_n|}{|a||a_n|}\\
&<\frac{2|a_n - a|}{|a|^2}\\
&<\frac{\epsilon|a|^2}{2} . (\frac{2}{|a|^2}) = \epsilon
\end{align*}

Note: If $n \geq n_1(\epsilon)$ then $|a_n - a| < \frac{|a|}{2}$ so $|a_n| > \frac{|a|}{2}$\\

vii) If $a > A$ then set $\epsilon = \frac{a-A}{2}$. Then $a - a_n \geq a-A > \epsilon \ \forall n$, so $a_n \cancel{\rightarrow} a$ \#\\

[Remark: $a_n < A, a_n \rightarrow a \cancel{\Rightarrow} a < A$, observe $-\frac{1}{n} < 0$ but $-\frac{1}{a_n} \rightarrow 0$]
\end{proof}

We need the following variation on the fundamental axiom.
\begin{AYExercise}
\label{Exercise 1.4}
A decreasing sequence of real numbers bounded below tends to a limit.
\end{AYExercise}

[Hint. If $a \leq b$ then $-b \leq -a$.]

\begin{proof}
If $b_1 \geq b_2 \geq b_3 \ldots $ and $b_n \geq B$ then $-b_1 \leq -b_2 \leq -b_3 \ldots$ and $-b_n \leq -B$ so by the fundamental axiom $-b_n \rightarrow \beta$, say, so $b_n = (-1).(-b_n) \rightarrow (-1).\beta = -\beta$
\end{proof}

Useful as the results of Lemma~\ref{SequenceLimitProperties} are, they are also true of sequences in $\mathbb{Q}$. They are therefore mere, if important, algebra. Our first truly `analysis' result may strike the reader as rather odd.

\begin{AYTheorem}[Axiom of Archimedes]
\label{ArchimedesAxiom}
\begin{equation*}
\frac{1}{n} \rightarrow 0\ \text{as $n\rightarrow\infty$}
\end{equation*}
\end{AYTheorem}

\begin{proof}
The sequence $1, \frac{1}{2}, \frac{1}{3}, \ldots$ is decreasing, and bounded below by 0, so $\frac{1}{n} \rightarrow l$ for some $l$.

Observe that $\frac{1}{2n}$ is a subsequence of $\frac{1}{n}$ so $\frac{1}{2n} \rightarrow l$.

But $\frac{1}{2n} = \frac{1}{2} \frac{1}{n} \rightarrow \frac{1}{2} l = \frac{l}{2}$

$\therefore l = \frac{l}{2} \Rightarrow l = 0$
\end{proof}

Theorem~\ref{ArchimedesAxiom} shows that there is no `exotic' real number $\gimel$ say (to choose an exotic symbol) with the property that $1/n > \gimel$ for all integers $n \geq 1$ yet $\gimel >0$ (that is, $\gimel$ is strictly positive and yet smaller than all strictly positive rationals). There exist number systems with such exotic numbers (the famous `non-standard analysis' of Abraham Robinson and the `surreal numbers' of Conway constitute two such systems) but, just as the rationals are, in some sense, too small a system for the standard theorems of analysis to hold so these non-Archimedean systems are, in some sense, too big. Archimedes and  Eudoxus realised the need for an axiom to show that there is no exotic number $\daleth$ bigger
than any integer\footnote{Footnote for passing historians, this is a course in mathematics.} (i.e. $\daleth > n$ for all integers $n\geq 1$; to see the connection with our form of the axiom consider $\gimel = 1/ \daleth$). However, in spite of its name, what was an axiom for Archimedes is a theorem for us.

\begin{AYTheorem} 
Given any real number $K$ we can find an integer $n$ with $n>K$.
\end{AYTheorem}

\begin{proof}
Assume we can find a $K$ that is greater than all numbers. Clearly it is greater than 1.

Thus $0 < \frac{1}{K} \leq \frac{1}{n}$, and thus $\frac{1}{n} \cancel{\rightarrow}0$\#

By reductio ad absurdum, $\cancel{\exists} K$ such that $K \geq n \ \forall n \in \mathbb{Z}$
\end{proof}

\section{Series and sums}

There is no need to restrict the notion of a limit to real numbers. 

\begin{AYDefinition}
\label{ComplexConvergenceDefinition}
If $a_n \in\mathbb{C}$ for each $n \geq 1$ and $a \in \mathbb{C}$ then we say that $a_n \rightarrow a$ if given $\epsilon > 0$ we can find an $n_0 (\epsilon)$ such that 
\begin{equation*}
|a_n-a| < \epsilon \text{ for all $n \geq n_0(\epsilon)$}
\end{equation*}
\end{AYDefinition}

\begin{AYExercise}
\label{ComplexSequencesExercise}
We work in $\mathbb{C}$.

(i) The limit is unique. That is, if $a_n \rightarrow a$ and $a_n \rightarrow b$ as $n \rightarrow \infty$ then $a = b$.

(ii) If $a_n \rightarrow a$ as $n \rightarrow \infty$ and $n(1) < n(2) < n(3) \ldots$ then $a_{n(j)} \rightarrow a$ as $j \rightarrow \infty$.

(iii) If $a_n = c$ for all $n$ then $a_n \rightarrow c$ as $n \rightarrow \infty$.

(iv) If $a_n \rightarrow a$ and $b_n \rightarrow b$ as $n \rightarrow \infty$ then $a_n + b_n \rightarrow a + b$.

(v) If $a_n \rightarrow a$ and $b_n \rightarrow b$ as $n \rightarrow \infty$ then $a_n b_n \rightarrow ab$.

(vi) If $a_n \rightarrow a$ as $n \rightarrow \infty$, $a_n \neq 0$ for each $n$ and $a \neq 0$, then $a_n^{-1} \rightarrow a^{-1}$.
\end{AYExercise}

\begin{AYExercise}
Explain why there is no result in Exercise~\ref{ComplexSequencesExercise} corresponding
to part (vii) of Lemma~\ref{SequenceLimitProperties}.
\end{AYExercise}

We illustrate some of the ideas introduced by studying infinite sums.

\begin{AYDefinition}
We work in $\mathbb{F}$ where $\mathbb{F}=\mathbb{R}$ or $\mathbb{F}=\mathbb{C}$. If $a_j \in \mathbb{F}$ we say that $\sum_{j=1}^\infty a_j$ converges to $s$ if
\begin{equation*}
\sum_{j=1}^N a_j \rightarrow s
\end{equation*}
as $N \rightarrow \infty$. We write $\sum_{j=1}^\infty a_j = s$.

If $\sum_{j=1}^N a_j$ does not tend to a limit as $N \rightarrow \infty$, we say that the sum $\sum_{j=1}^\infty a_j$
diverges.
\end{AYDefinition}

Note: Work as much as possible with $\sum_1^N a_n$ and as little as possible with $\sum_1^\infty a_n$ until you know that $\sum_1^\infty a_n$ exists\\

Example: Let $u_n = (-1)^{n+1}, S = \sum_1^\infty u_n$

i) $S = u_1 + \sum_{n=2}^\infty u_n = u_1 + \sum_{n=1}^\infty (-1) u_n = u_1 - \sum_{n=1}^\infty u_n = 1-S$ so $S = \frac{1}{2}$

ii) $S = \sum_{r=1}^\infty (u_{2r-1} + u_{2r}) = \sum_{r=1}^\infty (1-1) = \sum_{r=1}^\infty 0 = 0$

The above logic is faulty because $S$ does not exist

$\sum_1^N u_n = 1 - 1 + \cdots = \left\{\begin{array}{cl}
0 & \text{if }N \text{ even}\\
1 & \text{if }N \text{ odd}
\end{array}\right.$ does not converge

\begin{AYLemma}
We work in $\mathbb{F}$ where $\mathbb{F}=\mathbb{R}$ or $\mathbb{F}=\mathbb{C}$.

(i) Suppose $a_j, b_j \in \mathbb{F}$ and $\lambda, \mu \in \mathbb{F}$. If $\sum_{j=1}^\infty a_j$ and $\sum_{j=1}^\infty b_j$
converge then so does $\sum_{j=1}^\infty \lambda a_j + \mu b_j$ and
\begin{equation*}
\sum_{j=1}^\infty \lambda a_j + \mu b_j =\lambda \sum_{j=1}^\infty a_j + \mu \sum_{j=1}^\infty b_j
\end{equation*}

(ii) Suppose $a_j, b_j \in \mathbb{F}$ and there exists an $N$ such that $a_j = b_j$ for all $j \geq N$. Then, either $\sum_{j=1}^\infty a_j$ and $\sum_{j=1}^\infty b_j$ both converge or they both diverge. (In other words, initial terms do not matter.)
\end{AYLemma}

\begin{proof}
i) $\sum_{j=1}^N (\lambda a_j + \mu b_j) = \lambda \sum_{j=1}^N a_j + \mu \sum_{j=1}^N b_j \rightarrow \lambda \sum_{j=1}^\infty a_j + \mu \sum_{j=1}^\infty b_j$

ii) Suppose $a_j = b_j$ for $j \geq M$, then $a_j = b_j + c_j$ where $c_j = 0$ for $j \geq M$, so $\sum_1^N c_j = \sum_1^M c_j$ for $N \geq M \rightarrow \sum_1^M c_j$

So $\sum_1^\infty c_j$ exists, so if $\sum_1^\infty b_j$ exists, then $\sum_1^\infty (b_j + c_j)$ exists, $\therefore \sum_1^\infty a_j$ exists
\end{proof}

\begin{AYExercise}
Any problem on sums $\sum_{j=1}^\infty a_j$ can be converted into one on sequences by considering the sequence $s_n = \sum_{j=1}^n a_j$. Show conversely that a sequence $s_n$ converges if and only if, when we set $a_1 = s_1$ and $a_n = s_n - s_{n-1}$ $[n \geq 2]$ we have $\sum_{j=1}^\infty a_j$ convergent. What can you say about  $\lim_{n \rightarrow \infty} \sum_{j=1}^n a_j$ and $\lim_{n \rightarrow \infty} s_n$ if both exist?
\end{AYExercise}

The following results are fundamental to the study of sums. 
\begin{AYTheorem}[The comparison test]
\label{ComparisonTestTheorem}
We work in $\mathbb{R}$. Suppose that $0 \leq b_j \leq a_j$ for all $j$. Then, if $\sum_{j=1}^\infty a_j$ converges, so does
$\sum_{j=1}^\infty b_j$. 
\end{AYTheorem}

\begin{proof}
Observe that $\sum_{j=1}^N a_j$ is increasing, and recall that if $c_N \leq k \ \forall N$ and $c_N \rightarrow c$ then $c \leq k$, It follows that $\sum_{j=1}^N a_j \leq \sum_{j=1}^\infty a_j$

$0 \leq \sum_{j=1}^N b_j \leq \sum_{j=1}^N a_j \leq \sum_{j=1}^\infty a_j$. $\sum_{j=1}^N b_j$ is an increasing sequence, since $b_j \geq 0$

so since it is bounded above the fundamental theorem tells us that $\sum_{j=1}^N b_j$ tends to a limit as $n \rightarrow \infty$ i.e. $\sum_{j=1}^\infty b_j$ exists
\end{proof}

Note that was can use this an another proof of summing a geometric progression. Let $|x| < 1$

\begin{align*}
\sum_{n=0}^N x^n = \frac{1-x}{1-x} \sum_{n=0}^N x^n &= \frac{1 - x + x - x^2 + x^2 - \ldots - x^{N+1}}{1-x} \\
&= \frac{1 - x^{N+1}}{1-x} \rightarrow \frac{1}{1-x} \text{ as }N \rightarrow \infty
\end{align*}

Now look at $\sum_{n=1}^N \frac{x^n}{n^2 + 3 + \sin x}$ for $0 \leq x < 1$

Since $0 \leq \frac{x^n}{n^2 + 3 + \sin x} \leq x^n, \sum_{n=1}^\infty x^n$ exists, $\Rightarrow \sum_{n=1}^\infty \frac{x^n}{n^2 + 3 + \sin x}$ exists. 

\begin{AYTheorem}
\label{AbsoluteConvergenceTheorem}
We work in $\mathbb{F}$ where $\mathbb{F} = \mathbb{R}$ or $\mathbb{F} = \mathbb{C}$. If $\sum_{j=1}^\infty |a_j|$ converges, then so does $\sum_{j=1}^\infty a_j$.
\end{AYTheorem}

This is saying that, if we take a sequence, and plot the points of the sequence and join them up, $\sum_{j=1}^\infty |a_j|$ converges means the length of the line produced is bounded, and thus it converges to a point.

\begin{proof}
First consider it over $\mathbb{R}$

Set $a_j^+ = \left\{\begin{array}{cl}
a_j & \text{if } a_j \geq 0\\
0 & \text{if } a_j < 0
\end{array}\right.$

Set $a_j^- = \left\{\begin{array}{cl}
0 & \text{if } a_j \geq 0\\
-a_j & \text{if } a_j < 0
\end{array}\right.$
$\therefore a_j = a_j^+ - a_j^-$ and $0 \leq a_j^+ |a_j|, 0 \leq a_j^- \leq |a_j|$

$\sum_1^\infty a_j^+, \sum_1^\infty a_j^-$ converge because $\sum_1^\infty |a_j|$ does, using the comparison test. Thus

$\sum_{j=1}^\infty a_j = \sum_{j=1}^\infty (a_j^+ - a_j^-)$ convergses to $\sum_1^\infty a_j^+ - \sum_1^\infty a_j^-$

If $\mathbb{F} = \mathbb{C}$ with $a_j = x_j + i y_j, x_j, y_j \in \mathbb{R}$

$|x_j| \leq |a_j|$ so by comparison $\sum_1^\infty |x_j|$ converges, and so by the first part $\sum_1^\infty x_j$ also converges. Similarly $\sum_1^\infty y_j$ converges

So $\sum_{j=1}^\infty a_j = \sum_{j=1}^\infty (x_j = i y_j)$ converges to $\sum_{j=1}^\infty x_j + i \sum_{j=1}^\infty y_j$
\end{proof}

Theorem~\ref{AbsoluteConvergenceTheorem} is often stated using the following definition.

\begin{AYDefinition} We work in $\mathbb{F}$ where $\mathbb{F}=\mathbb{R}$ or $\mathbb{F}=\mathbb{C}$. If $\sum_{j=1}^\infty |a_j|$ converges we say that the sum $\sum_{j=1}^\infty a_j$ is absolutely convergent.
\end{AYDefinition}

Theorem~\ref{AbsoluteConvergenceTheorem} then becomes the statement that absolute convergence implies convergence.

Here is a trivial but useful consequence of Theorems~\ref{ComparisonTestTheorem} and~\ref{AbsoluteConvergenceTheorem}.

\begin{AYLemma}[Ratio test]
\label{RatioTestLemma}
Suppose $a_j \in \mathbb{C}$ and $|a_{j+1} / a_j| \rightarrow l$ as $j \rightarrow \infty$. If $l<1$, then $\sum_{j=1}^\infty |a_j|$ converges. If $l>1$, then $\sum_{j=1}^\infty a_j$ diverges.
\end{AYLemma}
Of course Lemma~\ref{RatioTestLemma} tells us nothing if $l=1$ or $l$ does not exist.

\begin{proof}
If $|\frac{a_{j+1}}{a_j}| \rightarrow l < 1$ then choose $\epsilon = \frac{1-l}{2}$. $\exists j(0)$ such that $\Big| |\frac{a_{j+1}}{a_j}| - l\Big| < \epsilon$ for $j \geq j(0)$

so $|\frac{a_{j+1}}{a_j} < l + \epsilon = \frac{1+l}{2} = K < 1$ for $j \geq j(0)$, so $|a_j| \leq |a_{j(0)}|K^{j - j(0)}$ for $j \geq j(0)$

We know that $\sum_1^\infty |a_{j(0)}K^{j - j(0)}$ converges (as it is a geometric progression with ratio $0 \leq K < 1$). We also know that the first $j(0)$ terms are irrelevant to convergence, and we know that
\begin{equation*}
|a_j| \leq |a_{j(0)}| K^{j - j(0)} \quad j \geq j(0)
\end{equation*}
so by comparison $\sum|a_j|$ converges

To prove the second part (if $|\frac{a_{n+1}}{a_n}| \rightarrow l$, $l > 1$ then $\sum a_n$) set $\epsilon = \frac{l-1}{2}$. $\exists N$ such that $\Big| |\frac{a_n + 1}{a_n}| - l\Big| < \epsilon = \frac{l-1}{2}$ for $n \geq N$, so $|\frac{a_n + 1}{a_n}| > \frac{l+1}{2}$ for $n \geq N$

$|a_n| > K(\frac{l+1}{2})^n$ for $n \geq N$, so $|a_n|$ unbounded, so $\sum a_n$ cannot converge
\end{proof}

Sums which are not absolutely convergent are much harder to deal with in general. It is worth keeping in mind the following trivial observation.

\begin{AYLemma}
We work in $\mathbb{F}$ where $\mathbb{F} = \mathbb{R}$ or $\mathbb{F} = \mathbb{C}$. If $\sum_{j=1}^\infty a_j$ converges, then
$a_j \rightarrow 0$ as $j \rightarrow \infty$.
\end{AYLemma}

\begin{proof}
Set $S_n = \sum_{j=1}^n a_j$. $a_{n+1} = s_{n+1} - s_n \rightarrow \sum_1^\infty a_j - \sum_1^\infty a_j = 0$
\end{proof}

At a deeper level the following result is sometimes useful.

\begin{AYLemma}[Alternating series test]
\label{AlternatingSeriesTest}
We work in $\mathbb{R}$. If we have a decreasing sequence of positive numbers $a_n$ with $a_n \rightarrow 0$ as $n \rightarrow \infty$, then $\sum_{j=1}^\infty (-1)^{j+1} a_j$ converges. Further
\begin{equation*}
\left|\sum_{j=1}^n (-1)^{j+1} a_j - \sum_{j=1}^\infty (-1)^{j+1} a_j \right| \leq |a_{N+1}|
\end{equation*}
for all $N\geq 1$.
\end{AYLemma}

\begin{proof}Set
\begin{align*}
S_1 & = a_1\\
S_2 & = a_1 - a_2\\
S_3 & = a_1 - a_2 + a_3\\
&\ \vdots
\end{align*}

Obeserve that $S_{2N + 1} = S_{2N-1} + (-a_{2N} + a_{2N + 1}) \leq S_{2N - 1}$ so the sequence $S_{2N+1}$ is decreasing. But also

\begin{align*}
S_{2N+1} &= a_1 - a_2 + a_3 - a_4 + \cdots\\
&= (a_1 - a_2) + (a_3 - a_4) + \ldots + (a_{2N-1} - a_{2N}) + a_{2N+1}\\
&\geq a_1 - a_2
\end{align*}
So $S_{2N+1}$ is a decreasing sequence bounded below, so $S_{2N+1} \rightarrow l$. Similarly $S_{2N}$ is an increasing sequence bounded above so $S_{2N} \rightarrow l'$. Now $a_{2N+1} = S_{2N+1} - S_{2N} \rightarrow l-l'$ and $a_{2N+1} \rightarrow 0$ so $l-l' = 0$ so $l = l'$ and $S_n \rightarrow l$ as $n \rightarrow \infty$

To finish observe that $S_{2N+1}$ is decreasing, $S_{2N}$ is increasing, so $S_{2N+1} \geq l \geq S_{2N}$ so $|l-S_{2N}| \leq |S_{2N} - S_{2N+1}| = a_{2N+1}$. Similarly $|l - S_{2N+1}| \leq a_{2N+2}$
\end{proof}

The last lemma is sometimes expressed by saying `the error caused by replacing a convergent infinite alternating sum of decreasing terms by the sum of its first few terms is no greater than the absolute value of the first term neglected'.

Later we will give another test for convergence called the integral test (Lemma~\ref{IntegralTestLemma}) from which we deduce the result known to many of you that $\sum_{n=1}^\infty n^{-1}$ diverges. I will give another proof in the next example.

\begin{AYExample}
(i) ${\displaystyle \sum_{n=1}^\infty \frac{1}{n}}$ diverges.

(ii) ${\displaystyle \sum_{n=1}^\infty \frac{(-1)^n }{n}}$ is
convergent but not absolutely convergent.

(iii) If $v_{2n} = 1/n$, $v_{2n-1} = -1/(2n)$ then $\sum_{n=1}^\infty v_n$ is not convergent.
\end{AYExample}

\begin{proof}
i) This shows that $a_n \rightarrow 0 \cancel{\rightarrow} \sum a_n$ converges

\begin{align*}
1 + \frac{1}{2} + \frac{1}{3} + \frac{1}{4} + \cdots &= 1 + \frac{1}{2} + (\frac{1}{3} + \frac{1}{4}) + (\frac{1}{5} + \frac{1}{6} + \frac{1}{7} + \frac{1}{8})\\
&\qquad + \cdots + (\frac{1}{2^{N-1} + 1} + \frac{1}{2^{N-1} + 2} + \cdots + \frac{1}{2^N}) + \cdots\\
&\geq 1 + \frac{1}{2} + (\frac{1}{4} + \frac{1}{4}) + (\frac{1}{8} + \frac{1}{8} + \frac{1}{8} + \frac{1}{8}) \\
&\qquad + \cdots + (\frac{1}{2^n} + \frac{1}{2^n} + \frac{1}{2^n} + \cdots + \frac{1}{2^n}) + \cdots\\
&\geq 1 + \frac{1}{2} + \frac{1}{2} + \frac{1}{2} + \cdots
\end{align*}
So $\sum_1^N \frac{1}{r}$ is not bounded and diverges

Note: $1 + \frac{1}{2} + \frac{1}{3} + \cdots = 1 + (\frac{1}{2} + \frac{1}{3}) + (\frac{1}{4} + \frac{1}{5} + \frac{1}{6} + \frac{1}{7}) + \cdots \leq 1 + (\frac{1}{2} + \frac{1}{2}) + (\frac{1}{4} + \frac{1}{4} + \frac{1}{4} + \frac{1}{4}) + \cdots + \frac{1}{2^N} \leq N+1$. Note that $2^20 \approx 10^6$, so $\sum_1^{10^6} \leq 21$ so it is very slow to diverge

ii) $\frac{1}{n}$ is decreasing, $\frac{1}{n} \rightarrow 0$ and $\frac{1}{n} \geq 0$ so we can apply the alternating series test. Thus $\sum \frac{(-1)^n}{n}$ is convergent, and i) tells us that is it not absolutely convergent.

iii) This is similar to i)
\begin{align*}v_1 + v_2 + \ldots + v_{2n} &= (v_1 + v_2) + (v_3 + v_4) + \cdots\\
&= (1 - \frac{1}{2}) + (\frac{1}{2} - \frac{1}{4}) + \cdots + (\frac{1}{n} - \frac{1}{2n})\\
&= \frac{1}{2} (1 + \frac{1}{2} + \frac{1}{3} + \cdots) \rightarrow \infty
\end{align*}
So $\sum v_j$ does not converge
\end{proof}

\section{Least upper bounds}
A non-empty bounded set in $\mathbb{R}$ need not have a maximum. 

\begin{AYExample}
The set $E= \{-1/n : n\geq 1\}$ is non-empty and any $e\in E$ satisfies the inequalities $-1 \leq e \leq 0$ but $E$ has no largest member.
\end{AYExample}

\begin{proof}
If $x \leq 0$ then $\exists n$ such that $-x > \frac{1}{n} > 0$ so $-\frac{1}{n} > x$ and $x$ is not a maximum

If $x \geq 0$ then $x \notin E$

Thus $E$ contains no largest member
\end{proof}

However, as we shall see every non-empty bounded set in $\mathbb{R}$ has a least upper bound (or supremum).

\begin{AYDefinition}
\label{SetSupremumDefinition}
Let $E$ be a non-empty set in $\mathbb{R}$. We say that $\alpha$ is a least upper bound for $E$ if

\begin{quote}
(i) $\alpha \geq e$ for all $e \in E$ [that is, $\alpha$ is an upper bound
for E] and
(ii) If $\beta \geq e$ for all $e \in E$ then $\beta \geq \alpha$ [that is, $\alpha$ is the least such upper bound]
\end{quote}
\end{AYDefinition}

If $E$ has a supremum $\alpha$ we write $\sup_{e \in E} e= \sup E = \alpha$.

\begin{AYLemma}
\label{UniqueSupremumLemma}
If the least upper bound exists it is unique.
\end{AYLemma}

\begin{proof}
Suppose $E$ is a non-empty subset of $\mathbb{R}$ and $\alpha, \alpha'$ are least upper bounds for $E$. Then since $\alpha'$ is an upper bound, and $\alpha$ is a least upper bound $\alpha \leq \alpha'$. Similarly $\alpha' \leq \alpha$ and so $\alpha' = \alpha$
\end{proof}

The following remark is trivial but sometimes helpful.

\begin{AYLemma}
\label{LeastSupremumExists}
Let $E$ be a non-empty set in $\mathbb{R}$. Then $\alpha$ is a least upper bound for $E$ if and only if we can find $e_n \in E$ with $e_n \rightarrow \alpha$ and $b_n$ such that $b_n \geq e$ for all $e \in E$ and $b_n \rightarrow \alpha$ as $n \rightarrow \infty$.
\end{AYLemma}

\begin{proof}
First show that $\alpha$ is an upper bound.

If $e \in E$ then $b_n \geq e$. But $b_n \rightarrow \alpha$ so $\alpha \geq e$.

Then show $\alpha$ is the least upper bound i.e. if $\beta$ is an upper bound $\beta \geq \alpha$

If $\beta$ is an upper bound then, in particular, $\beta \geq e_n$, but $e_n \rightarrow \alpha$ so $\beta \geq \alpha$
\end{proof}

Here is the promised result.

\begin{AYTheorem}
\label{UBImpliesLUB}
Any non-empty set in $\mathbb{R}$ with an upper bound has a least upper bound.
\end{AYTheorem}

\begin{proof}
Let $E$ be such a set. Since $E$ is non-empty, we can find $a_0 \in E$. Observe that $\exists e \in E$ such that $e \geq a_0$ (e.g. $a_0$ itself). Since $E$ is bounded, we can find $b_0$ such that $b_0 \geq e \ \forall e \in E$. Set $c_0 = \frac{a_0 + b_0}{2}$. 

Either $\exists e \in E$ such that $e \geq c_0$ and we put $a_1 = c_0, b_1 = b_0$

Or $\forall e \in E \ c_0 > e$ and we put $a_1 = a_0, b_1 = c_0$

Observe that 

\begin{quote}
$a_0 \leq a_1 \leq b_1 \leq b_0$

$b_1 - a_1 = \frac{b_0 - a_0}{2^1}$ and 

1) $\exists e_1 \in E$ such that $e_1 \geq a_1$

2) $b_1 \geq e \ \forall e \in E$
\end{quote}

Repeating we obtain

\begin{quote}
$a_0 \leq a_1  \leq \ldots \leq a_n \leq b_n \leq \ldots \leq b_1 \leq b_0$

$b_n - a_n = \frac{b_0 - a_0}{2^n}$ and 

1) $\exists e_n \in E$ such that $e_n \geq a_1$

2) $b_n \geq e \ \forall e \in E$
\end{quote}

The $a_n$s form an increasing sequence bounded above, so $a_n \rightarrow \alpha$ by the fundamental theorem

$b_n = a_n + (b_n + a_n) = a_n + 2^{-n} (b_0 - a_0) \rightarrow \alpha + 0 = \alpha$ so $a_n \leq e_n \leq b_n$ with all the $e_n \in E$, so $e_n \rightarrow \alpha$ and $b_n \geq e \ \forall e \in E$ so $\alpha$ is the least upper bound of $E$
\end{proof}

We observe that this result is actually equivalent to the fundamental axiom.

\begin{AYTheorem}
Theorem~\ref{UBImpliesLUB} implies the fundamental axiom.
\end{AYTheorem}

\begin{proof}
On the assumption that every bounded non-empty set has a least upper bound, suppose $a_1 \leq a_2 \leq \ldots \leq a_n \leq A_0 \forall n$

Set $A = \{a_1, a_2, \ldots \}$ is non-empty and bounded, so it has a supremum which we call $\alpha$. Given $\epsilon > 0$ we know $\alpha - \epsilon$ is not an upper bound, so $\exists N$ such that $a_N > \alpha - \epsilon$. Now if $n \geq N, \alpha \geq a_n \geq a_N \geq \alpha - \epsilon$, so $|a_n - \alpha| < \epsilon$ and $a_n \rightarrow \alpha$
\end{proof}

Of course we have the notion of a greatest lower bound or infimum.

\begin{AYExercise}
\label{InfimumExercise}
Define the greatest lower bound in the manner of Definition~\ref{SetSupremumDefinition}, prove its uniqueness in the manner of Lemma~\ref{UniqueSupremumLemma} and state and prove a result corresponding to Lemma~\ref{LeastSupremumExists}.
\end{AYExercise}

If $E$ has an infimum $\beta$ we write $\inf_{e \in E} e = \inf E = \beta$. One way of dealing with the infimum is to use the following observation.

\begin{AYLemma}
\label{NegativeInfimumLemma}
Let $E$ be a non-empty set in $\mathbb{R}$ and write $-E=\{-e : e \in E\}$. Then $E$ has an infimum if and only if $-E$ has a supremum. If $E$ has an infimum $\inf E = -\sup (-E)$.
\end{AYLemma}

\begin{proof}
If $-E$ has a supremum $\alpha$

i) $\alpha \geq -e \ \forall e \in E$

ii) If $\beta \geq -e \ \forall e \in E$ then $\beta \geq \alpha$

So 

i)' $-a \leq e \ \forall e \in E$

ii)' If $-\beta \leq e \ \forall e \in E$ then $-\beta \leq -\alpha$

So $-\alpha$ is the infimum of $E$
\end{proof}

\begin{AYExercise}
Use Lemma~\ref{NegativeInfimumLemma} and Theorem~\ref{UBImpliesLUB} to show that any non-empty set in $\mathbb{R}$ with a lower bound has a greatest lower bound. 
\end{AYExercise}

The notion of a supremum will play an important  r\^{o}le in our proofs of Theorem~\ref{BoundedContinuousAttainsBounds} and Theorem~\ref{ConvergenceRadiusTheorem}.

The following result is also equivalent to the fundamental axiom (that is, we can deduce it from the fundamental axiom and conversely, if we take it as an axiom, rather than a theorem, then we can deduce the fundamental axiom as a theorem).

\begin{AYTheorem}[Bolzano-Weierstrass]
\label{BolzanoWeierstrassTheorem}
If $x_n \in \mathbb{R}$ and there exists a $K$ such that $|x_n| \leq K$ for all $n$, then we can find $n(1) < n(2) < \ldots$ and $x \in \mathbb{R}$ such that $x_{n(j)} \rightarrow x$ as $j \rightarrow \infty$.
\end{AYTheorem}

\begin{proof}
Set $a_0 = -K, b_0 = K$. $\{m : x_m \in [a_0, b_0]\}$ is infinite. Set $c_0 = \frac{a_0 + b_0}{2}$

If $\{m : x_m \in [a_0, c_0]\}$ is infinite we set $a_1 = a_0, b_1 = c_0$, otherwise set $a_1 = c_0, b_1 = b_0$

Observe that
\begin{quote}
$a_0 \leq a_1 \leq b_1 \leq b_0$

$b_1 - a_1 = 2^{-1} (b_0 - a_0)$

$\{m : x_m \in [a_1, b_1]\}$ is infinite
\end{quote}

Repeat. $a_n$ is increasing and bounded above so tends to a limit, call it $\alpha$

$b_n = a_n + (b_n - a_n) = a_n + (b_0 - a_0)2^{-n} \rightarrow \alpha$

Since $[a_n, b_n]$ contains an $x_m$ for infinitely many $m$ I can choose $m(1) < m(2) < m(3) < \ldots$ such that $x_{m(j)} \in [a_j, b_j]$. Thus $a_j \leq x_{m(j)} \leq b_j$ and as $a_j, b_j \rightarrow \alpha$, $x_{m(j)} \rightarrow \alpha$
\end{proof}

The Bolzano-Weierstrass theorem says that every bounded sequence of reals has a convergent subsequence. Notice that we say nothing about uniqueness; if $x_n=(-1)^n $ then $x_{2n}\rightarrow 1$ but $x_{2n+1}\rightarrow -1$ as $n\rightarrow\infty$.

We proved the theorem of Bolzano-Weierstrass by `lion hunting' but your supervisor may well show you another method. We shall use the Bolzano-Weierstrass theorem to prove that every continuous function on a closed bounded interval is bounded and attains its bounds (Theorem~\ref{BoundedContinuousAttainsBounds}). The Bolzano-Weierstrass theorem will be much used in the next analysis course because it generalises to many dimensions.

Note also that the Bolzano-Weierstrass theorem implies the fundamental axiom

\begin{proof}
Suppose $a_1 \leq a_2 \leq \ldots$ and $a_n \leq A$. Then $\{a_j\}$ is a bounded sequence so by Bolzano-Weierstrass it has a convergent subset i.e. $\exists n(1) < n(2) < \ldots$ such that $a_{n(k)} \rightarrow \alpha$

Since $a_{n(k)} \leq a_j \ \forall j \geq n(k)$ we have $\alpha \geq a_n(k) \ \forall k$ so $\alpha \geq a_j \ \forall j$

Now given $\epsilon > 0 \ \exists n(k)$ such that $|\alpha - a_{n(k)}| < \epsilon$ so $\alpha \geq a_j \geq a_{n(k)} \ \forall j \geq n(k)$, so $|\alpha - a_j| < \epsilon \ \forall j \geq n(k)$ and thus $a_j \rightarrow \alpha$
\end{proof}

\begin{figure}[htbp]
	\centering
		\includegraphics{Images/AY01.jpeg}
	\caption{Equivalence of fundamental axioms}
	\label{Equivalence of fundamental axioms}
\end{figure}

\section{Continuity}

We make the following definition.

\begin{AYDefinition}
\label{ContinuityDefinitionOne}
A function $f: \mathbb{R} \rightarrow \mathbb{R}$ is continuous at $x$ if given $\epsilon > 0$ we can find a $\delta (\epsilon, x) > 0$ [read `a delta depending on epsilon and $x$'] such that
\begin{equation*}
|f(x)-f(y)| < \epsilon
\end{equation*}
for all $y$ with $|x-y| < \delta(\epsilon, x)$.

If $f$ is continuous at each point $x \in \mathbb{R}$ we say that $f$ is a continuous function on $\mathbb{R}$.
\end{AYDefinition}

I shall do my best to make this seem a reasonable definition but it is important to realise that I am really stating a rule of the game (like a knights move in chess or the definition of offside in football). If you wish to play the game you must accept the rules. Results about continuous functions must be derived from the definition and not stated as `obvious from the notion of continuity'.

In practice we use a slightly more general definition.

\begin{AYDefinition}
\label{ContinuityDefinitionTwo}
Let $E$ be a subset of $\mathbb{R}$. A function $f: E \rightarrow \mathbb{R}$ is continuous at $x \in E$ if given $\epsilon > 0$ we can find a $\delta(\epsilon, x) > 0$ [read `a delta depending on epsilon and $x$'] such that 
\begin{equation*}
|f(x)-f(y)| < \epsilon
\end{equation*}
for all $y \in E$ with $|x-y| <\delta(\epsilon, x)$.

If $f$ is continuous at each point $x \in E$, we say that
$f$ is a continuous function on $E$.
\end{AYDefinition}

However, it will do no harm and may be positively helpful if, whilst you are getting used to the idea of continuity, you concentrate on the case $E = \mathbb{R}$.

\begin{AYLemma}
\label{ContinuityProperties}
Suppose that $E$ is a subset of $\mathbb{R}$, that $x \in E$, and that $f$ and $g$ are functions from $E$ to $\mathbb{R}$.

(i) If $f(x) = c$ for all $x \in E$, then f is continuous on $E$.

(ii) If $f$ and $g$ are continuous at $x$, then so is $f + g$.

(iii) Let us define $f \times g : E \rightarrow \mathbb{R}$ by $f \times g(t) = f(t) g(t)$ for all $t \in E$. Then if $f$ and $g$ are continuous at $x$, so is $f \times g$.

(iv) Suppose that $f(t) \neq 0$ for all $t \in E$. If $f$ is continuous at $x$ so is $1/f$.
\end{AYLemma}

\begin{proof}
i) Trivial

ii) As $f, g$ are continous at $x$

Given $\epsilon > 0 \ \exists \delta_1(\epsilon) > 0$ such that $|f(x) - f(y)| < \epsilon$ for $|x-y| < \delta_1(\epsilon)$

Given $\epsilon > 0 \ \exists \delta_2(\epsilon) > 0$ such that $|g(x) - g(y)| < \epsilon$ for $|x-y| < \delta_2(\epsilon)$

Now set $\delta_3(\epsilon) = \min \{\delta_1(\frac{\epsilon}{2}), \delta_2(\frac{\epsilon}{2})\}$

Then if $|x - y| < \delta_3(\epsilon)$
\begin{align*}
|(f + g)(x) - (f + g)(y)| &= |f(x) + g(x) - f(y) + g(y)|\\
&= |(f(x) - f(y)) + (g(x) - g(y))|\\
&\leq |f(x) - f(y)| + |g(x) - g(y)|\\
&\leq \frac{\epsilon}{2} + \frac{\epsilon}{2} = \epsilon
\end{align*}

iii) Is done similarly

iv) $f$ is continuous at $x$

Thus given $\epsilon > 0 \ \exists \delta(\epsilon) > 0$ such that $|f(x) - f(y)| < \epsilon$ for $|x-y| < \delta(\epsilon)$

Set $\delta_1(\epsilon) = \min\{\delta(\frac{|f(x)|}{2}), \delta(\frac{\epsilon |f(x)|^2}{2})\}$

Then if $|x-y| < \delta_1(\epsilon)$
\begin{align*}
|\frac{1}{f(x)} - \frac{1}{f(y)}| &\leq \frac{|f(y) - f(x)|}{|f(x)||f(y)|}\\
&\leq \frac{2|f(x) - f(y)|}{|f(x)|^2|}\\
&< \epsilon
\end{align*}
Using $|f(y) - f(x)| < \frac{|f(x)|}{2}$ so $|f(y)| > \frac{|f(x)|}{2}$
\end{proof}

\begin{AYLemma}
\label{ContinuousChainRule} 
Let $U$ and $V$ be subsets of $\mathbb{R}$. Suppose $f : U \rightarrow \mathbb{R}$ is such that $f(t) \in V$ for all $t \in U$. If $f$ is continuous at $x \in U$ and $g : V \rightarrow \mathbb{R}$ is continuous at $f(x)$, then the composition $g \circ f$ is continuous at $x$.
\end{AYLemma}

\begin{proof}
Given $\epsilon > 0 \ \exists \ \delta_1(\epsilon) > 0$ such that $|f(x) - f(y)| < \epsilon$ for $|x-y| < \delta_1(\epsilon)$

Given $\epsilon > 0 \ \exists \ \delta_2(\epsilon) > 0$ such that $|g(f(x)) - g(u)| < \epsilon$ for $|u-f(x)| < \delta_2(\epsilon)$

Set $\delta_3(\epsilon) = \delta_1 (\delta_2(\epsilon))$

Then if $|x-y| < \delta_3(\epsilon)$ we have $|f(x) - f(y)| < \delta_2(\epsilon)$ and so $|g(f(x)) - g(f(y))| < \epsilon$
\end{proof}

By repeated use of parts~(ii) and~(iii) of Lemma~\ref{ContinuityProperties} it is easy to show that polynomials $P(t)=\sum_{r=0}^n a_r t^{r}$ are continuous. The details are spelled out in the next exercise.

\begin{AYExercise}
Prove the following results.

(i) Suppose that $E$ is a subset of $\mathbb{R}$ and that $f : E \rightarrow \mathbb{R}$ is continuous at $x \in E$. If $x \in  E' \subset E$ then the restriction $f|_{E'}$ of $f$ to $E'$ is also continuous at $x$.

(ii) If $J : \mathbb{R} \rightarrow \mathbb{R}$ is defined by $J(x) = x$ for all $x \in \mathbb{R}$, then $J$ is continuous on $\mathbb{R}$.

(iii) Every polynomial $P$ is continuous on $\mathbb{R}$.

(iv) Suppose that $P$ and $Q$ are polynomials and that $Q$ is never zero on some subset $E$ of $\mathbb{R}$. Then the rational function $P/Q$ is continuous on $E$ (or, more precisely, the restriction of $P/Q$ to $E$ is continuous.)
\end{AYExercise}

The following result is little more than an observation but will be very useful.

\begin{AYLemma}
Suppose that $E$ is a subset of $\mathbb{R}$, that $x \in E$, and that $f$ is continuous at $x$. If $x_n \in E$ for all $n$ and $x_n \rightarrow x$ as $n \rightarrow \infty$, then $f(x_n) \rightarrow f(x)$ as $n \rightarrow \infty$.
\end{AYLemma}

\begin{proof}
Given $\epsilon > 0 \ \exists \ \delta(\epsilon) > 0$ such that $|f(x) - f(y)| < \epsilon$ for $|x-y| < \delta(\epsilon)$

Given $\epsilon > 0 \ \exists \ N(\epsilon)$ such that $|x_n - x| < \epsilon$ for $n \geq N(\epsilon)$

If we set $M(\epsilon) = N(\delta(\epsilon))$ then $n \geq M(\epsilon) \Rightarrow |x_n - x| < \delta(\epsilon) \Rightarrow |f(x_n) - f(x)| < \epsilon$
\end{proof}

So far in this section we have only done algebra but the next result depends on the fundamental axiom. It is one of the key results of analysis and although my recommendation runs contrary to a century of enlightened pedagogy I can see no objections to students learning the proof as a model. Notice that the theorem resolves the problem posed by Example~\ref{RationalExample}~(i).

\begin{AYTheorem}[The intermediate value theorem]
\label{IntermediateValueTheorem}
If $f : [a,b] \rightarrow \mathbb{R}$ is continuous and $f(a) \geq 0 \geq f(b)$ then there exists a $c \in [a, b]$ such that $f(c)=0$
\end{AYTheorem}

\begin{proof}
This relies on the fact that we are working in $\mathbb{R}$

Set $a_0 = a, b_0 = b, c_0 = \frac{a_0 + b_0}{2}$

If $f(c_0) \geq 0$ set $a_1 = c_0, b_1 = b_0$, otherwise set $a_1 = a_0, b_1 = c_0$. Observe that 

\begin{quote}
$a_0 \leq a_1 \leq b_1 \leq b_0$

$b_1 - a_1 = 2^{-1} (b_0 - a_0)$

$f(a_1) \geq 0 \geq f(b_1)$
\end{quote}

Repeat, obtaining

\begin{quote}
$a_0 \leq a_1 \leq \ldots \leq a_n \leq b_n \leq \ldots \leq b_1 \leq b_0$

$b_n - a_n = 2^{-n} (b_0 - a_0)$

$f(a_n) \geq 0 \geq f(b_n)$
\end{quote}

$a_n$ is an increasing sequence bounded above, so $a_n \rightarrow \alpha$ with $\alpha \leq b_n \forall n$ as $a_n \leq b_n$

$b_n = a_n + (b_n - a_n) = a_n + 2^{-n} (b_0 - a_0) \rightarrow \alpha + 0 = \alpha$, and as $b_n \geq a_n \ \forall n, \alpha \geq a$

Now $a_n \rightarrow \alpha$ so $f(a_n) \rightarrow f(\alpha)$ and similarly $f(b_n) \rightarrow f(\alpha)$. But $f(a_n) \geq 0$ so $f(\alpha) \geq 0$ and $f(b_n) \leq 0$ so $f(\alpha) \leq 0$. Thus $f(\alpha) = 0$
\end{proof}

The next three exercises are applications of the intermediate value theorem.

\begin{AYExercise}
Show that any real polynomial of odd degree has at least one root. Is the result true for polynomials of even degree? Give a proof or counterexample.
\end{AYExercise}

\begin{proof}
Let $p(x) = x^3 + ax^2 + bx + c = x^3(\frac{a}{x} + \frac{b}{x^2} + \frac{c}{x^3}) \geq x^3 (1 - \frac{|a|}{x} - \frac{|b|}{x^2} - \frac{|c|}{x^3}) > \frac{x^5}{4}$

If we set $x \geq 4(2 + |a| + |b| + |c|), p(x) \geq 0$ and for $x$ sufficiently large and negative $p(x)$ is negative, so by continuity has a root.
\end{proof}

\begin{AYExercise}
Suppose that $g : [0,1] \rightarrow [0,1]$ is a continuous function. By considering $f(x) = g(x)-x$, or otherwise, show that there exists a $c \in [0,1]$ with $g(c) = c$. (Thus every continuous map of $[0,1]$ into itself has a fixed point.) Give an example of a bijective (but, necessarily, non-continuous) function $h : [0,1] \rightarrow [0,1]$ such that $h(x) \ neq x$ for all $x \in [0,1]$.

\noindent [Hint: First find a function $H : [0,1] \setminus \{0, 1, 1/2 \} \rightarrow [0,1] \setminus \{0, 1, 1/2 \}$ such that $H(x) \neq x$.]
\end{AYExercise}

\begin{proof}
Let $f(x) = g(x) - x$

$f(0) \geq 0$ (if $f(0) = 0, 0 = c$ and done)

$f(1) \leq 0$ (if $f(1) = 0, 0 = c$ and done)
Therefore using the Intermediate Value Theorem $\exists c$ such that $f(c) = 0$ and thus $g(c) = c$
\end{proof}

\begin{AYExercise}
Every mid-summer day at six o'clock in the morning, the youngest monk from the monastery of Damt starts to climb the narrow path up Mount Dipmes. At six in the evening he reaches the small temple at the peak where he spends the night in meditation. At six o'clock in the morning on the following day he starts downwards, arriving back at the monastery at six in the evening. Of course, he does not always walk at the same speed. Show that, none the less, there will be some time of day when he will be at the same place on the path on both his upward and downward journeys.
\end{AYExercise}

We proved the intermediate value theorem (Theorem~\ref{IntermediateValueTheorem}) by lion hunting. We prove the next two theorems by using the Bolzano-Weierstrass Theorem (Theorem~\ref{BolzanoWeierstrassTheorem}). Again the results are very important and I can see no objection to learning the proofs as a model. 

\begin{AYTheorem}
If $f : [a,b] \rightarrow \mathbb{R}$ is continuous then we can find an $M$ such that $|f(x)| \leq M$ for all $x \in [a,b]$.
\end{AYTheorem}

\begin{proof}
Note that this needs to be on a closed interval, on $(0, 1) \ f: x \mapsto \frac{1}{x}$ is unbounded

Suppose $f$ is unbounded

Then we can find $x_n \in [a, b]$ such that $|f(x_n)| \geq n$

The $x_n$ lie in a bounded interval so they have a convergent subsequence $x_{n(j)} \rightarrow \alpha$ say. Since $x_{n(j)} \leq b, \alpha \leq b$ and since $x_{n(j)} \geq a, \alpha \geq a$ so $\alpha \in [a, b]$

Now $f$ is continuous at $\alpha$ so $\exists \ \delta > 0$ such that $|f(\alpha) - f(x)| < 1 \ \forall |x-\alpha| < \delta$

$x \in [a, b]$ so $|f(x)| \leq |f(\alpha) + 1| \ \forall |x-\alpha| < \delta, x \in [a, b]$

But $x_{n(j)} \rightarrow \alpha$. So $\exists J$ such that if $j \geq J \ |x_{n(j)} - \alpha| < \delta$

so $n(j) \leq |f(x_{n(j)}| \leq |f(\alpha)| + 1 \ \forall j \geq J$ \#

Thus $f$ must be bounded
\end{proof}

In other words a continuous function on a closed bounded interval is bounded. We improve this result in the next theorem.

\begin{AYTheorem}
\label{BoundedContinuousAttainsBounds}
If $f : [a,b] \rightarrow \mathbb{R}$ is continuous then we can find $x_1, x_2 \in [a,b]$ such that
\begin{equation*}
f(x_1) \leq f(x) \leq f(x_2)
\end{equation*}
for all $x\in[a,b]$.
\end{AYTheorem}

\begin{proof}
Start by observing that $\{f(x) : x \in [a, b]\}$ is a non-empty bounded set, so it has a supremum $M$. What we need to show is that $\exists \alpha \in [a, b]$ such that $f(\alpha) = M$

Since $M$ is the supremum of $\{f(x) : x \in [a, b]\}$ we can certainly find $x_n \in [a, b]$ such that $f(x_n) \geq M - \frac{1}{n}$ (if not, then $(M - \frac{1}{n})$ would be an upper bound, but $M$ is the least upper bound). Automatically $M \geq f(x_n)$

By Bolzano-Weierstrass there is a convergenet subsequence $x_{n(j)} \rightarrow \alpha$. As before $\alpha \in [a, b]$ and $f(x_{n(j)}) \rightarrow f(\alpha)$. But $M \geq f(x_{n(j)}) \geq M - \frac{1}{n(j)}$, so $f(x_{n(j)}) \rightarrow M$ so $f(\alpha) = M$
\end{proof}

In other words a continuous function on a closed bounded interval is bounded and attains its bounds. 

\section{Differentiation}

In this section it will be useful to have another type of limit.

\begin{AYDefinition}
Let $E$ be a subset of $\mathbb{R}$, $f$ be some function from $E$ to $\mathbb{R}$, and $x$ some point of $E$. If $l \in \mathbb{R}$ we say that $f(y) \rightarrow l$ as $y \rightarrow x$ [or, if we wish to emphasise the restriction to $E$
that $f(y) \rightarrow l$ as $y \rightarrow x$ through values $y \in E$] if, given $\epsilon > 0$, we can find a $\delta(\epsilon)>0$ [read `a delta depending on epsilon'] such that
\begin{equation*}
|f(y)-l|<\epsilon
\end{equation*}
for all $y \in E$ with $0 < |x-y| < \delta(\epsilon)$.
\end{AYDefinition}

As before there is no real loss if the reader initially takes $E=\mathbb{R}$.

The following two exercises are easy but useful. 

\begin{AYExercise}
Let $E$ be a subset of $\mathbb{R}$. Show that a function $f : E \rightarrow \mathbb{R}$ is continuous at $x \in E$ if and only if  $f(y) \rightarrow f(x)$ as $y \rightarrow x$. 
\end{AYExercise}

\begin{AYExercise}
Let $E$ be a subset of $\mathbb{R}$, $f$, $g$ be some functions from $E$ to $\mathbb{R}$, and $x$ some point of $E$. 

(i) The limit is unique. That is, if $f(y) \rightarrow l$ and $f(y) \rightarrow k$ as $y \rightarrow x$ then $l=k$.

(ii) If $x \in E' \subseteq E$ and $f(y) \rightarrow l$ as $y \rightarrow x$ through values $y \in E$, then $f(y) \rightarrow l$
as $y \rightarrow x$ through values $y \in E'$.

(iii) If $f(t) = c$ for all $t \in E$ then $f(y) \rightarrow c$ as $y \rightarrow x$.

(iv) If $f(y) \rightarrow l$ and $g(y) \rightarrow k$ as $y \rightarrow x$ then $f(y) + g(y) \rightarrow l+k$.

(v) If $f(y) \rightarrow l$ and $g(y) \rightarrow k$ as $y \rightarrow x$ then $f(y) g(y) \rightarrow lk$.

(vi) If $f(y) \rightarrow l$ as $y \rightarrow x$, $f(t) \neq 0$ for each $t \in E$ and $l \neq 0$ then $f(t)^{-1} \rightarrow l^{-1}$.

(vii) If $f(t) \leq L$ for each $t \in E$ and $f(y)\rightarrow l$  as $y\rightarrow x$ then $l\leq L$.
\end{AYExercise}

\begin{proof}
iii) $f(t) = c \ \forall t$. Given $\epsilon > 0$ set $\delta = 1$. $|f(y) - c| = |c-c| = |0| = 0 < \epsilon$

iv) Since $f(y) \rightarrow l, g(y) \rightarrow k$ as $y \rightarrow x$

Given $\epsilon > 0 \ \exists \ \delta_1(\epsilon) > 0$ such that $|f(y) - l| < \epsilon$ for $|y-x| < \delta_1(\epsilon)$

Given $\epsilon > 0 \ \exists \ \delta_2(\epsilon) > 0$ such that $|g(y) - k| < \epsilon$ for $|y-x| < \delta_2(\epsilon)$

Thus given $\epsilon > 0$ if we set $\delta_3(\epsilon) = \min \{(\delta_1(1), \delta_1(\frac{\epsilon}{2(1 + |k|)}), \delta_2(\frac{\epsilon}{2(1 + |l|)})\}$ then

\begin{align*}
|f(y) g(y) - lk| &= |f(y) g(y) - f(y) k + f(y) k - lk|\\
&\leq |f(y) g(y) - f(y) k| + |f(y) k - lk|\\
&= |f(y)||g(y) - k| + |f(y) - l||k|\\
&\leq (|l| + 1)|g(y) - k| + |f(y) - l||k|
\end{align*}
\end{proof}

We can now define the derivative.

\begin{AYDefinition}
Let $E$ be a subset of $\mathbb{R}$. A function $f : E \rightarrow \mathbb{R}$ is differentiable at $x \in E$ with derivative $f'(x)$
if 
\begin{equation*}
\frac{f(y)-f(x)}{y-x}\rightarrow f'(x)
\end{equation*}
as $y \rightarrow x$.

If $f$ is differentiable at each point $x \in E$, we say that $f$ is a differentiable function on $E$. 
\end{AYDefinition}
As usual, no harm will be done if you replace $E$ by $\mathbb{R}$.

You can think of differentiation as 
\begin{equation*}
f(y) = \underbrace{f(x) + f'(x)(y-x)}_{\text{linear}} + \text{error term}
\end{equation*}

to be useful the error has to $\rightarrow 0$ faster than the linear term.

Note: We can also use this in 3-D with grad

\begin{align*}
\delta f &= \bigtriangledown f . \delta x\\
\delta f &= \frac{\delta f}{\delta x_1} \delta x_1 + \frac{\delta f}{\delta x_2} \delta x_2 + \frac{\delta f}{\delta x_3} \delta x_3\\
\Rightarrow f(\underline{y}) &= f(\underline{x}) + \bigtriangledown f_0 . (\underline{y} - \underline{x}) + \text{error term}
\end{align*}
and as before error $\rightarrow 0$ faster than the linear term

Here are some easy consequences of the definition.

\begin{AYExercise} 
Let $E$ be a subset of $\mathbb{R}$, $f$ some function from $E$ to $\mathbb{R}$, and $x$ some point of $E$. Show that if $f$ is differentiable at $x$ then $f$ is continuous at $x$.
\end{AYExercise}

\begin{proof}
Given $\epsilon > 0 \ \exists \ \delta(\epsilon) > 0$ such that $|\frac{f(x) - f(y)}{y-x} - f'(x)| < \epsilon$ for $|y - x| < \delta(\epsilon)$, so

\begin{align*}
|f(y) - f(x) - f'(x)(y-x)| &< \epsilon|y-x|\\
\text{thus }|f(y) - f(x)| &< \epsilon |y-x| + |f'(x) (y-x)|\\
&= \epsilon |y-x| + |f'(x)||y-x|
\end{align*}

All of the terms in the last expression are small except $|f'(x)|$ which is fixed. So if we set $\delta_1(\epsilon) = \min\{\delta(1), \frac{\epsilon}{2}, \frac{\epsilon}{2(|f'(x) + 1|)}\}$ for $\epsilon < 1$. Then if $0 < |x-y| < \delta_1(\epsilon)$

\begin{align*}
|f(x) - f(y)| &< 1 . |y-x| + |f'(x)||y-x|\\
&< \frac{\epsilon}{2} + \frac{\epsilon}{2}\\
&= \epsilon
\end{align*}
\end{proof}

\begin{AYExercise}
Let $E$ be a subset of $\mathbb{R}$, $f$, $g$ be some functions from $E$ to $\mathbb{R}$, and $x$ some point of $E$. Prove the following results.

(i) If $f(t) = c$ for all $t \in E$ then $f$ is differentiable at $x$ with $f'(x) = 0$.

(ii) If $f$ and $g$ are differentiable at $x$ then so is their sum $f + g$ and 
\begin{equation*}
(f+g)'(x) = f'(x) + g'(x)
\end{equation*}

(iii) If $f$ and $g$ are differentiable at $x$ then so is their product $f \times g$ and 
\begin{equation*}
(f \times g)'(x) = f'(x)g(x) + f(x)g'(x)
\end{equation*}

(iv) If $f$ is differentiable at $x$ and $f(t) \neq 0$ for all $t \in E$ then $1/f$ is differentiable at $x$ and
\begin{equation*}
(1/f)'(x) = -f'(x)/f(x)^{2}
\end{equation*}

(v) If $f(t)=\sum_{r=0}^n a_r t^{r}$ on $E$ then $f$ is differentiable at $x$ and
\begin{equation*}f'(x)=\sum_{r=1}^n ra_r x^{r-1}
\end{equation*}
\end{AYExercise}

\begin{proof}
iii) \begin{align*}
\frac{f(y) g(y) - f(x) g(x)}{y - x} &= \frac{f(y) (g(y) - g(x))}{y-x} + \frac{g(x) (f(y) - f(x))}{y-x}\\
&= f(x) g'(x) + g(x) f'(x)
\end{align*}
(Note: We have used that, as we pass to the limit, $f(y) \rightarrow f(x)$)

iv) \begin{align*}
\frac{\frac{1}{f(y)} - \frac{1}{f(x)}}{y-x} &= \frac{1}{f(y)f(x)} . \frac{f(x) - f(y)}{y-x}\\
&= -\frac{1}{f(y) f(x)} . \frac{f(y) - f(x)}{y-x}\\
&\rightarrow -\frac{f'(x)}{(f(x))^2}
\end{align*}
\end{proof}
The next result is slightly harder to prove than it looks. (we split the proof into two halves depending on whether $f'(x)\neq 0$ or $f'(x)=0$).

\begin{AYLemma}
Let $U$ and $V$ be subsets of $\mathbb{R}$. Suppose $f : U \rightarrow \mathbb{R}$ is such that $f(t) \in V$ for all $t \in U$. If $f$ is differentiable at $x \in U$ and $g : V \rightarrow \mathbb{R}$ is differentiable at $f(x)$, then the composition $g \circ f$ is differentiable at $x$ with 
\begin{equation*}
(g \circ f)'(x)=f'(x)g'(f(x))
\end{equation*}
\end{AYLemma}

\begin{proof}
If $f'(x) \neq 0$ then $f(x+h) - f(x) \neq 0$ for $h$ small and thus

\begin{align*}
\frac{g(f(x+h)) - g(f(x))}{h} &= \frac{g(f(x+h)) - g(f(x))}{f(x+h) - f(x)} . \frac{f(x+h) - f(x)}{h}\\
&= g'(f(x)). f'(x)
\end{align*}
(since $f$ is continuous at $x \leq 0$, $f(x+h) \rightarrow f(x)$)

We have lots of different ways of dealing with $f'(x) = 0$. Here is one

Since $g$ is differentiable at $f(x)$
\begin{equation*}
\frac{g(f(x) + h) - g(f(x))}{k} \rightarrow g'(f(x))
\end{equation*}
so if $k$ is sufficiently small, say for $0 < |k| < \mu$
\begin{equation*}
\Big|\frac{g(f(x) + k) - g(f(x))}{k} - g'(f(x))\Big| < 1
\end{equation*}
So $|g(f(x) + k) - g(f(x))| \leq |k| (1 + |g'(f(x))|)$

Now $f$ is continuous at $x$ so $\exists \delta(\mu) > 0$ such that $|f(x+h) - f(x)| < \mu$ for $|h| < \delta(\mu)$

Take $k = f(x+h) - f(x)$. We have

\begin{equation*}
|g(f(x+h)) - g(f(x))| \leq |f(x+h) - f(x)|(1 + |g'(f(x))|)
\end{equation*}
so
\begin{equation*}
\Big|\frac{g(f(x+h)) - g(f(x))}{h}\Big| \leq \underbrace{\Big|\frac{f(x+h) - f(x)}{h}\Big|}_{\rightarrow 0} (1 + |g'(f(x))|)
\end{equation*}

Thus $g \circ f$ is differentiable with derivative 0
\end{proof}

\section{The mean value theorem}
We have almost finished our project of showing that the horrid situation revealed by Example~\ref{RationalExample} can not occur for the reals.

Our first step is to prove Rolle's theorem.
\begin{AYTheorem}[Rolle's theorem]
If $g : [a,b] \rightarrow \mathbb{R}$ is a continuous function with $g$ differentiable on $(a,b)$ and $g(a) = g(b)$, then we can find a $c \in (a,b)$ such that $g'(c) = 0$.
\end{AYTheorem}

\begin{proof}
A continuous function on a closed bounded interval is bounded and attains its bounds (by theorem \ref{BoundedContinuousAttainsBounds})

In other words, we can find $c_1, c_2 \in [a, b]$ such that $g(c_1) \geq g(x) \geq g(c_2) \ \forall x \in [a, b]$

If both are end points (i.e. $c_1, c_2 \in \{a, b\}$) then $g(c_1) = g(c_2)$ so $g(x) = g(c_1) \ \forall x \in [a, b]$ so $g$ is constant and we could take $c = \frac{a+b}{2}$. Hence we may assume that at least one of $c_1, c_2$ is not an end point.

WLOG assume that $c_1$ is not an end point (otherwise consider $-g$

Let $c_1 = c$, we have $(c-\delta, c + \delta) \subset [a, b]$ for some $\delta > 0$

Observe that $f(c+h) - g(c) \leq 0$ (as $c$ is a maximum point)

So, if $h > 0 \quad \frac{g(c+h) - g(c)}{h} \leq 0$ so allowing $h \rightarrow 0$ through positive values $g'(c) \leq 0$

Similarly if $h < 0 \quad \frac{g(c+h) - g(c)}{h} \geq 0$ so allowing $h \rightarrow 0$ through negative values $g'(c) \geq 0$. 

Thus $g'(c) = 0$
\end{proof}

A simple tilt gives the famous mean value theorem.

\begin{AYTheorem}[The mean value theorem]
\label{MeanValueTheorem}
If $f : [a,b] \rightarrow \mathbb{R}$ is a continuous function with $f$ differentiable on $(a,b)$, then we can find a $c\in (a,b)$ such that
\begin{equation*}
f(b)-f(a)=(b-a)f'(c)
\end{equation*}
\end{AYTheorem}

\begin{proof}
Set $g(t) = (f(t) - f(a)) - \frac{t-a}{t-b} (f(b) - f(a))$

$g(t) = A f(t) + B$ where $A, B$ are such that $g(a) = g(b) = 0$. As $g$ is differentiable on $(a, b)$ and continuous on $[a, b]$, by Rolle's Theorem $\exists c \in (a, b)$ such that $g'(c) = 0$ i.e. $0 = f'(c) - \frac{f(b) - f(a)}{b-a}$
\end{proof}

We now have the results so long desired.

\begin{AYLemma}
\label{DifferentiableFunctionProperties}
If $f : [a,b] \rightarrow \mathbb{R}$ is a continuous function with $f$ differentiable on $(a,b)$, then the following results hold.

(i) If $f'(t) > 0$ for all $t \in (a,b)$ then $f$ is strictly increasing on $[a,b]$. (That is, $f(y) > f(x)$ whenever $b \geq y > x\geq a$.)

(ii) If $f'(t)\geq 0$ for all $t\in (a,b)$ then $f$ is increasing on $[a,b]$. (That is, $f(y)\geq f(x)$ whenever $b\geq y>x\geq a$.)

(iii) {\bf [The constant value theorem]} If $f'(t)=0$ for all $t\in (a,b)$ then $f$ is constant on $[a,b]$. (That is, $f(y)=f(x)$ whenever $b \geq y > x \geq a$.)
\end{AYLemma}

\begin{proof}
i) $f(y) - f(x) = (y-x)f'(c)$ with $c$ between $x$ and $y$ by the MVT, and thus as $f'(c)$ is positive and $y-x$ is $f(y) > f(x)$

ii) Same as above, except that $f'(c)$ might be 0, so can get $\geq$ instead of $>$

iii) $f(y) - f(x) = (y-x) f'(c) = 0$

The partical converses are easy:

If $f$ is differentiable, and increasing, then $\frac{f(x+h) - f(x)}{h} \geq 0 \ \forall h \neq 0$ so 
\begin{equation*}
f'(x) = \lim_{h \rightarrow 0} \frac{f(x+h) - f(x)}{h} \geq 0
\end{equation*}
\end{proof}

Notice that since we deduce Lemma~\ref{DifferentiableFunctionProperties} from the mean value theorem we can not use it in the proof of Rolle's theorem.

The mean value theorem has many important consequences, some of which we look at in the remainder of the section.

We start by looking at inverse functions.

\begin{AYLemma}
(i) Suppose $f : [a,b] \rightarrow \mathbb{R}$ is continuous. Then $f$ is injective if and only if it is strictly increasing (that is $f(t) > f(s)$ whenever $a \leq s < t \leq b$) or strictly decreasing.

(ii) Suppose $f : [a,b] \rightarrow \mathbb{R}$ is continuous and strictly increasing. Let $f(a) = c$ and $f(b) = d$. Then the map $f : [a,b] \rightarrow [c,d]$ is bijective and $f^{-1}$ is continuous on $[c,d]$. 
\end{AYLemma}

\begin{proof}
If $f$ is strictly increasing, $y \neq x$ then either $y > x$ or $x > y$. If $y > x, f(y) > f(x)$ so $f(y) \neq f(x)$, same for $x > y$

Conversely, suppose $f$ is not strictly increasing. Then either
\begin{quote}
A) We can find $x_1 < x_2 < x_3$ such that $f(x_1) \leq f(x_2)$ and $f(x_3) \leq f(x_2)$

B) we can find $x_1 < x_2 < x_3$ such that $f(x_1) \geq f(x_2)$ and $f(x_3) \geq f(x_2)$
\end{quote}

WLOG assume case A (otherwise look at $-f$)

If $f(x_1) = f(x_2)$ or $f(x_2) = f(x_3)$ then we have a contradiction ($f$ would not be injective). So suppose all are strict inequalities

Choose $c$ with $f(x_2) > c > \max\{f(x_1), f(x_2)\}$. By the Intermediate Value Theorem we can find $\alpha_1, \alpha_2$ such that $x_1 < \alpha_1 < x_3, x_2 < \alpha_2 < x_3$ and $f(\alpha_1) = x = f(\alpha_2)$. Thus $f$ is not injective

ii) The injectivity follows from i)

Surjectivity follows from the Intermediate Value Theorem: If $x \leq \gamma \leq d$ then since $f(a) = c, f(b) = d \ \exists t$ with $a \leq t \leq b$ such that $f(t) = \gamma$

Thus $f$ is bijective and $f^{-1}$ is defined as a function, $f^{-1} : [c, d] \mapsto [a, b]$

Claim that $f^{-1}$ is continuous

Let $y \in [c, d]$. To simplify matters, suppose $y \neq c, y \neq d$

If $\delta > 0$ consider the interval $[\theta_1, \theta_2] = [a, b] \cap [f^{-1}(y - \frac{\delta}{2}), f^{-1}(y + \frac{\delta}{2})]$

$\theta_1 < f^{-1}(y) < \theta_2$

$f(\theta_1) < y < f(\theta_2)$ so choose $\epsilon > 0$ such that $f(\theta_1) < y - \epsilon < y < y + \epsilon < f(\theta_2)$

If $|y-z| < \epsilon$, $f(\theta_1) < z < f(\theta_2)$ so $\theta_1 < f^{-1}(z) < \theta_2$ and so $|f^{-1}(z) - f^{-1}(y)| < \frac{\delta}{2} < \delta$

The proof if $y = c$ or $y = d$ is similar, but one-sided
\end{proof}

\begin{AYLemma}[Inverse rule]
Suppose $f : [a,b] \rightarrow \mathbb{R}$ is differentiable on $[a,b]$ and $f'(x) > 0$ for all $x \in [a,b]$. Let $f(a) = c$ and $f(b) = d$. Then the map $f : [a,b] \rightarrow [c,d]$ is bijective and $f^{-1}$ is differentiable on $[c,d]$ with
\begin{equation*}
f^{-1}{}'(x)=\frac{1}{f'(f^{-1}(x))}
\end{equation*}
\end{AYLemma}

\begin{proof}
\begin{equation*}
\frac{f^{-1}(y + h) - f^{-1}(y)}{h} = \frac{f^{-1}(y + h) - f^{-1}(y)}{f(f^{-1}(y+h)) - f(f^{-1}(y))}
\end{equation*}
But $f$ is continuous, so $f^{-1}(y + h) - f^{-1}(y) \rightarrow 0$ and thus
\begin{equation*}
\frac{f^{-1}(y + h) - f^{-1}(y)}{h} \rightarrow \frac{1}{f'(f^{-1}(y))}
\end{equation*}
\end{proof}

Note: Suppose we know that $f$ and $f^{-1}$ are differentiable, then $t = f(f^{-1}(t)) = f \circ f^{-1}(t)$. By the chain rule $1 = f'(f^{-1}(t)) . (f^{-1})'(t)$ so $(f^{-1})'(t) = \frac{1}{f'(f^{-1}(t))}$

This does not constitute a proof!

In the opinion of the author the true meaning of the inverse rule and the chain rule only becomes clear when we consider higher dimensions in the next analysis course.

We now prove a form of Taylor's theorem. 

\begin{AYTheorem}[$n$th mean value theorem]
Suppose that $b > a$ and $f : [a,b] \rightarrow \mathbb{R}$ is $n+1$ times differentiable. Then 
\begin{equation*}
f(b)-\sum_{j=0}^n \frac{f^{(j)}(a)}{j!} (b-a)^{j}= \frac{f^{(n+1)}(c')}{(n+1)!} (b-a)^{n+1}
\end{equation*}
for some $c'$ with $a < c' < b$.
\end{AYTheorem}

\begin{proof}
This is complicated, so we try to simplify a little

Step 1: By translation $x \mapsto x-a$ we may suppose $a = 0$, and our statement becomes
\begin{equation*}
f(b) - \sum_{n=0}^n \frac{f^{(j)}(0)}{j!} b^j = \frac{f^{(n+1)}(c')}{(n+1)!} b^{n+1}
\end{equation*}
with $0 < c' < b$

Step 2: Set $g(t) = f(t) - \sum_{r=0}^n \frac{f^{(r)} (0)}{r!} t^r$

Then $g(0) = g'(0) = \cdots = g^{(n)}(0) = 0$ and our theorem becomes

If $g$ is $(n+1)$ times differentiable, and $g(0) = g'(0) = \cdots = g^{(n)}(0) = 0$ then
\begin{equation*}
g(b) = \frac{g^{(n+1)}(c)}{(n+1)!} b^{n+1} \tag{$g$ theorem}
\end{equation*}
We now need to prove the $g$ theorem

Let $G(x) = g(x) - \frac{x^{n+1}}{b^{n+1}} g(b)$. Then $G(0) = G'(0) = \cdots = G^{(n)}(0) = 0$ and $G(b) = 0$, so by Rolle, $\exists 0 < b_1 < b$ such that $G'(b_1) = 0$

And because $G'(0) = 0 \ \exists 0 < b_2 < b_1$ such that $G''(b_2) = 0$

And because $G''(0) = 0 \ \exists 0 < b_3 < b_2$ such that $G'''(b_3) = 0$ and so on

Until we get to $0 < b_n < b_{n-1}$ with $G^{(n)}(b_n) = 0$. But $G^{(n)}(0) = 0$ so $\exists 0 < c < b_n$ such that $G^{(n+1)}(c) = 0$

\begin{equation*}
G^{(n+1)}(x) = g^{(n+1)}(c) - \frac{g(b) (n+1)!}{b^n} \Rightarrow g(b) = \frac{g^{(n+1)}(c) b^{n+1}}{(n+1)!}
\end{equation*}
\end{proof}

This gives us a global and a local Taylor's theorem.

\begin{AYTheorem}[Global Taylor theorem]
Suppose that $b > a$ and $f : [a,b] \rightarrow \mathbb{R}$ is $n+1$ times differentiable. If $x,\, t\in[a,b]$
\begin{equation*}
f(t) = \sum_{j=0}^n \frac{f^{(j)}(x)}{j!} (t-x)^{j} + \frac{f^{(n+1)}(x+\theta(t-x))}{(n+1)!} (t-x)^{n+1}
\end{equation*}
for some $\theta \in (0,1)$.
\end{AYTheorem}

\begin{AYTheorem}[Local Taylor theorem]
\label{LocalTaylorTheorem}
Suppose that $\delta>0$ and $f:(x-\delta,x+\delta)\rightarrow\mathbb{R}$ is $n$ times differentiable on $(x-\delta,x+\delta)$ and $f^{(n)}$ is continuous at $x$. Then
\begin{equation*}
f(t) = \sum_{j=0}^n \frac{f^{(j)}(x)}{j!} (t-x)^{j} + \epsilon(t)(t-x)^n
\end{equation*}
where $\epsilon(t) \rightarrow 0$ as $t \rightarrow x$.
\end{AYTheorem}

\begin{proof}
Move to the origin, $f(t) = f(0) + f'(0)t + \cdots + \frac{f^{(n-1)}(0)}{(n-1)!} t^{n-1} + \frac{f^{(n)}(c)t^n}{n!}$

If $f^{(n)}$ is continuous at O
\begin{align*}
f^{(n)}(c) - f^{(n)}(0) \rightarrow &\text{as }t \rightarrow 0\\
&\text{so }c \rightarrow 0
\end{align*}

Write $\epsilon(t) = (f^{(n)}(c) - f^{(n)}(0))\frac{1}{n!}$

\begin{equation*}
f(t) = f(0) + f'(0)t + \cdots + \frac{f^{(n-1)}(0) t^{n-1}}{(n-1)!} + \frac{f^{(n)}(0) t^n}{n!} \epsilon(t) t^n
\end{equation*}
\end{proof}

Notice that the local Taylor theorem always gives us some information but that the global one is useless unless we can find a useful bound on the $n+1$th derivative.

To reinforce this warning we consider a famous example of Cauchy. 

\begin{AYExercise}
\label{InitialJumpExample}
Consider the function $F:\mathbb{R}\rightarrow\mathbb{R}$ defined by
\begin{align*}
F(0) &= 0\\
F(x) &= \exp(-1/x^{2}) \qquad \text{otherwise}.
\end{align*}

(i) Prove by induction, using the standard rules of differentiation, that $F$ is infinitely differentiable at all points $x \neq 0$ and that, at these points, 
\begin{equation*}
F^{(n)}(x) = P_n (1/x) \exp(-1/x^{2})
\end{equation*}
where $P_n$ is a polynomial which need not be found explicitly.

(ii) Explain why $x^{-1} P_n(1/x) \exp(-1/x^{2}) \rightarrow 0$ as $x\rightarrow 0$.

(iii) Show by induction, using the definition of differentiation, that $F$ is infinitely differentiable at $0$ with $F^{(n)}(0)=0$ for all $n$. [Be careful to get this part of the argument right.]

(iv) Show that
\begin{equation*}
F(x)=\sum_{j=0}^\infty\frac{F^{(j)}(0)}{j!}x^{j}
\end{equation*}
if and only if $x=0$. (The reader may prefer to say that `The Taylor expansion of $F$ is only valid at $0$'.)

(v) Why does part~(iv) not contradict the local Taylor theorem (Theorem~\ref{LocalTaylorTheorem})?
\end{AYExercise}

Since examiners are fonder of the global Taylor theorem than it deserves I shall go through the following example. 

\begin{AYExample}
Assuming the standard properties of the exponential function show that
\begin{equation*}
\exp x=\sum_{j=0}^\infty\frac{x^{j}}{j!}
\end{equation*}
for all $x$.
\end{AYExample}

\begin{proof}
If $f(x) = e^x$ then $f^{(r)}x = e^x$ and using the formula
\begin{equation*}
f(x) = f(0) + \frac{f'(0)}{1!}x + \cdots + \frac{f^{(n)}(0)}{n!} x^n + \frac{f^{(n+1)}(c)}{(n+1)!} x^{n+1} \quad |c| < |x|
\end{equation*}
which becomes 

\begin{equation*}
e^x = 1 + \frac{x}{1!} + \frac{x^2}{2!} + \cdots + \frac{x^n}{n!} + e^c \frac{x^{n+1}}{(n+1)!}
\end{equation*}

Look at $R_n = e^c \frac{x^{n+1}}{(n+1)!}$, the error term

$|R_n| \leq e^{|x|}\frac{|x|^{n+1}}{(n+1)!} \rightarrow 0$ (because if $n \geq 2|x| + 1, \frac{|x|}{n} \leq \frac{1}{2}$

So $e^x = 1 + x + \frac{x^2}{2!} + \cdots$
\end{proof}

Please note that in a pure mathematics question many (or even most) of the marks in a question of this type will depend on estimating the remainder term. [In methods questions you may simply be asked to `find the Taylor's series' without being asked to prove convergence.]

\section{Complex variable}

The field $\mathbb{C}$ of complex numbers resembles the field $\mathbb{R}$ of real numbers in many ways but not in all. 

\begin{AYLemma}
We can not define an order on $\mathbb{C}$ which will behave in the same way as $>$ for $\mathbb{R}$.
\end{AYLemma}

However there is sufficient similarity for us to define limits, continuity and differentiability. (We have already seen some of this in  Definition~\ref{ComplexConvergenceDefinition} and Exercise~\ref{ComplexSequencesExercise}.)

\begin{AYDefinition}
Let $E$ be a subset of $\mathbb{C}$, $f$ be some function from $E$ to $\mathbb{C}$, and $z$ some point of $E$. If $l \in \mathbb{C}$ we say that $f(w) \rightarrow l$ as $w \rightarrow z$ [or, if we wish to emphasise the restriction to $E$ that $f(w) \rightarrow l$ as $w \rightarrow z$ through values $w \in E$] if, given $\epsilon > 0$, we can find a $\delta(\epsilon)>0$ [read `a delta depending on epsilon'] such that
\begin{equation*}
|f(w)-l| < \epsilon
\end{equation*}
for all $w \in E$ with $0 < |w-z| < \delta(\epsilon)$.
\end{AYDefinition}

As usual there is no real loss if the reader initially takes $E=\mathbb{C}$.

\begin{AYDefinition}
Let $E$ be a subset of $\mathbb{C}$. We say that a function $f : E \rightarrow \mathbb{C}$ is continuous at $z \in E$ if and only if $f(w) \rightarrow f(z)$ as $w \rightarrow z$. 
\end{AYDefinition}

\begin{AYExercise}
Let $E$ be a subset of $\mathbb{C}$, $f$, $g$ be some functions from $E$ to $\mathbb{C}$, and $z$ some point of $E$.

(i) The limit is unique. That is, if $f(w) \rightarrow l$ and $f(w) \rightarrow k$ as $w \rightarrow z$ then $l=k$.

(ii) If $z \in E' \subseteq E$ and $f(w) \rightarrow l$ as $w \rightarrow z$ through values $w \in E$, then $f(w) \rightarrow l$ as $w \rightarrow x$ through values $w \in E'$.

(iii) If $f(u) = c$ for all $u \in E$ then $f(w) \rightarrow c$ as $w \rightarrow z$.

(iv) If $f(w) \rightarrow l$ and $g(w) \rightarrow k$ as $w \rightarrow z$ then $f(w) + g(w) \rightarrow l + k$.

(v) If $f(w) \rightarrow l$ and $g(w) \rightarrow k$ as $w \rightarrow z$ then $f(w) g(w) \rightarrow lk$.

(vi) If $f(w) \rightarrow l$ as $w \rightarrow z$, $f(u) \neq 0$ for each $u \in E$ and $l \neq 0$ then $f(w)^{-1} \rightarrow l^{-1}$.
\end{AYExercise}

\begin{AYExercise}
Suppose that $E$ is a subset of $\mathbb{C}$, that $z \in E$, and that $f$ and $g$ are functions from $E$ to $\mathbb{C}$.

(i) If $f(u) = c$ for all $u \in E$, then f is continuous on $E$.

(ii) If $f$ and $g$ are continuous at $z$, then so is $f + g$.

(iii) Let us define $f \times g : E \rightarrow \mathbb{C}$ by $f \times g(u) = f(u) g(u)$ for all $u \in E$. Then if $f$ and $g$ are continuous at $z$, so is $f \times g$.

(iv) Suppose that $f(u) \neq 0$ for all $u \in E$. If $f$ is continuous at $z$ so is $1/f$.

(v) If $z \in E' \subset E$ and $f$ is continuous at $z$ then the restriction $f|_{E'}$ of $f$ to $E'$ is also continuous at $z$.

(vi) If $J : \mathbb{C} \rightarrow \mathbb{C}$ is defined by $J(z) = z$ for all $z \in \mathbb{C}$, then $J$ is continuous on $\mathbb{C}$.

(vii) Every polynomial $P$ is continuous on $\mathbb{C}$.

(viii) Suppose that $P$ and $Q$ are polynomials and that $Q$ is never zero on some subset $E$ of $\mathbb{C}$. Then the rational function $P/Q$ is continuous on $E$ (or, more precisely, the restriction of $P/Q$ to $E$ is continuous.) 
\end{AYExercise}

\begin{AYExercise}
Let $U$ and $V$ be subsets of $\mathbb{C}$. Suppose $f : U \rightarrow \mathbb{C}$ is such that $f(z) \in V$ for all $z \in U$. If $f$ is continuous at $w \in U$ and $g : V \rightarrow \mathbb{C}$ is continuous at $f(w)$, then the composition $g \circ f$ is continuous at $w$.
\end{AYExercise}

\begin{AYDefinition}
Let $E$ be a subset of $\mathbb{C}$. A function $f : E \rightarrow \mathbb{C}$ is differentiable at $z \in E$ with derivative $f'(z)$ if 
\begin{equation*}
\frac{f(w)-f(z)}{w-z}\rightarrow f'(z)
\end{equation*}
as $w \rightarrow z$.

If $f$ is differentiable at each point $z\in E$ we say that $f$ is a differentiable function on $E$.
\end{AYDefinition}

\begin{AYExercise}
Let $E$ be a subset of $\mathbb{C}$, $f$ some function from $E$ to $\mathbb{C}$, and $z$ some point of $E$. Show that if $f$ is differentiable at $z$ then $f$ is continuous at $z$.
\end{AYExercise}

\begin{AYExercise}
Let $E$ be a subset of $\mathbb{C}$, $f$, $g$ be some functions from $E$ to $\mathbb{C}$, and $z$ some point of $E$. Prove the following results.

(i) If $f(u)=c$ for all $u\in E$ then $f$ is differentiable at $z$ with $f'(z)=0$.

(ii) If $f$ and $g$ are differentiable at $z$ then so is their sum $f + g$ and 
\begin{equation*}
(f + g)'(z) = f'(z) + g'(z)
\end{equation*}

(iii) If $f$ and $g$ are differentiable at $z$ then so is their product $f \times g$ and 
\begin{equation*}
(f \times g)'(z)=f'(z) g(z) + f(z) g'(z)
\end{equation*}

(iv) If $f$ is differentiable at $z$ and $f(u) \neq 0$ for all $u \in E$ then $1/f$ is differentiable at $z$ and
\begin{equation*}
(1/f)'(z)=-f'(z)/f(z)^{2}
\end{equation*}

(v) If $f(u) = \sum_{r=0}^n a_r u^{r}$ on $E$ then $f$ is differentiable at $z$ and
\begin{equation*}
f'(z)=\sum_{r=1}^n ra_r z^{r-1}
\end{equation*}
\end{AYExercise}

\begin{proof}
v) If $f_n(z) = z^n$ then $f$ is differentiable with $f_n' = nf_{n-1}$

So prove by induction, already proved for 0, 1

Suppose true for $n$. $f_{n+1} = f_n . f_1$ so by the multiplicative rule

$f_{n+1}' = f_n'.f_1 + f_n . f_1' = nf_{n-1} . f_1 + f_n . 1 = (n+1)f_n$
\end{proof}

\begin{AYExercise}[Chain rule]
Let $U$ and $V$ be subsets of $\mathbb{C}$. Suppose $f : U \rightarrow \mathbb{C}$ is such that $f(z) \in V$ for all $t \in U$. If $f$ is differentiable at $w \in U$ and $g : V \rightarrow \mathbb{R}$ is differentiable at $f(w)$, then the composition $g \circ f$ is differentiable at $w$ with 
\begin{equation*}
(g \circ f)'(w) = f'(w)g'(f(w))
\end{equation*}
\end{AYExercise}

In spite of these similarities the subject of  complex differentiable functions is very different from that of real differentiable functions. It turns out that `well behaved' complex functions need not be differentiable.

\begin{AYExample}
Consider the map $\Gamma : \mathbb{C} \rightarrow \mathbb{C}$ given by $\Gamma(z)=\overline{z}$. The function $\Gamma$ is nowhere differentiable.
\end{AYExample}

\begin{proof}
\begin{equation*}
\frac{\Gamma(z + h) - \Gamma(z)}{h} = \frac{\overline{(z + h)} - \overline{z}}{h} = \frac{\overline{h}}{h}
\end{equation*}

Claim that $\frac{\overline{h}}{h}$ does not tend to a limit as $h \rightarrow 0$

First, let $h \in \mathbb{R}, h \rightarrow 0$. $\frac{\overline{h}}{h} = \frac{h}{h} = 1 \rightarrow 1$

Now let $h = ik, j \in \mathbb{R}, k \rightarrow 0$. $\frac{\overline{h}}{h} = \frac{-ik}{ik} = -1 \rightarrow -1$ \#
\end{proof}

We can view this another way (this is not a proof)

\begin{align*}
\frac{\overline{(z + h)} - \overline{z}}{h} &= \frac{\overline{(z + re^{i \theta})} - \overline{z}}{re^{i \theta}}\\
&= \frac{re^{-i\theta}}{re^{i\theta}}\\
&= e^{-2i\theta}
\end{align*}
so the derivative depends on the direction taken

$f(z + iw) = f(z) + Re^{i\theta} + \text{error}$

Also, given that $\overline{z}$ changes sense it is not surprising that it is not differentiable

Because complex differentiability is so much more restrictive than real differentiability we can prove stronger theorems about  complex differentiable functions. For example it can be shown that such functions can be written locally  as power series\footnote{The syllabus says that this fact is part of this course. In this one instance I advise you to ignore the syllabus.} (contrast the situation in the real case revealed by Example~\ref{InitialJumpExample}). To learn more go to the course~P3 on complex methods. 

\section{Power series}

In this section we work in $\mathbb{C}$ unless otherwise stated. We start with a very useful observation.

\begin{AYLemma}
If $\sum_{n=0}^\infty a_n z_{0}^n$ converges and $|z| < |z_{0}|$ then $\sum_{n=0}^\infty a_n z^n$ converges. 
\end{AYLemma}

\begin{proof}
Since $\sum a_n z_0^n$ converges, $a_n z^n \rightarrow 0$. Since the series $a_n z_0^n \rightarrow 0$ it is bounded i.e. $\exists K$ such that $|a_n z_0^n| \leq K \ \forall n \geq 1$

(this follows: $a_n z_0^n \rightarrow 0 \Rightarrow \exists N : |a_n z_0^n| \leq 1 \ \forall n \geq N$, take $K = \max\{1, |a_0|, |a_1z|, |a_2 z_0^2|, \ldots, |a_N z_0^N|\}$

Thus $|a_n z^n| = |a_n z_0^n| |\frac{z}{z_0}|^n \leq K|\frac{z}{z_0}|^n$

Since $|\frac{z}{z_0}| \leq 1, \sum|\frac{z}{z_0}|^n$ converges over the reals (as it is a geometric progression) (note, we say nothing about $\mathbb{C}$), so by the comparison test $\sum |a_n z^n|$ converges, and since an absolutely convergent series converges, $\sum a_n z^n$ converges
\end{proof}

This gives us the following basic theorem on power series.

\begin{AYTheorem}
\label{ConvergenceRadiusTheorem}
Suppose that $a_n \in \mathbb{C}$. Then either $\sum_{n=0}^\infty a_n z^n $ converges for all $z \in \mathbb{C}$, or there exists a real number $R$ with $R \geq 0$ such that

(i) $\sum_{n=0}^\infty a_n z^n $ converges if $|z| < R$,

(ii) $\sum_{n=0}^\infty a_n z^n $ diverges if $|z| > R$.
\end{AYTheorem}

\begin{proof}
If $\sum a_n z^n$ converges $\forall z$, we set $R = \infty$, nothing to prove.

If not, consider $E = \{|z| : \sum a_n z_n $diverges$\}$

$E$ is a subset of $\mathbb{R}$, $E \neq \phi$, $E$ bounded below by 0. $R = \in E$ is thus defined. We must now prove that it has the required properties.

If $|z| < R$ then $\sum a_n z^n$ converges by definition

If $|z| > R$ then by definition we can find $|w| \in \mathbb{C}$ such that $|z| > |w|$ and $\sum a_n w^n$ diverges. Now, by the previous lemma, if $\sum a_n z^n$ converged then $\sum a_n w^n$ would converge, but it does not, so $\sum a_n z^n$ diverges
\end{proof}

We call $R$ the radius of convergence of $\sum_{n=0}^\infty a_n z^n$. If $\sum_{n=0}^\infty a_n z^n$ converges for all $z$ we write $R = \infty$. 

Warning: Although the ratio test is a useful first test for trying to find $R$ it is not always successful and it cannot be used in the definition. Look at

$1 + 2z + z^2 + 2z^3 + z^4 + 2z^5 + \cdots$

$|\frac{a_{n+1}}{a_n}|$ is alternately $2 |z|$ and $\frac{|z|}{2}$

However, $1 + z^2 + z^4 + \cdots$ converges, and $2z + 2z^3 + 2z^5 + \cdots = 2z(1 + z^2 + z^4 + \cdots)$ converges too (if $|z| < 1$, so $1 + 2z + z^2 + 2z^3 + z^4 + 2z^5 + \cdots$ converges for $|z|<1$

$|z|^n \rightarrow \infty$ if $|z| > 1$, so $1 + 2z + z^2 + 2z^3 + z^4 + \cdots$ must diverge if $|z| > 1$, so the radius of convergence if 1

We never say anything about behaviour on the radius of convergence.

The following useful strengthening is left to the reader as an exercise.

\begin{AYExercise}
Suppose that $\sum_{n=0}^\infty a_n z^n $ has radius of convergence $R$. Then the sequence $|a_n z^n |$ is unbounded if $|z|>R$
and $\sum_{n=0}^\infty a_n z^n $ converges absolutely if $|z|<R$.
\end{AYExercise}
Note that we say nothing about what happens on the circle of convergence. 

\begin{AYExample}

(i) $\sum_{n=1}^\infty n^{-2}z^n$ has radius of convergence $1$ and converges for all $z$ with $|z|=1$.

(ii) $\sum_{n=1}^\infty z^n $ has radius of convergence $1$ and diverges for all $z$ with $|z|=1$. 
\end{AYExample}

\begin{proof}
i) If $|z| = 1, |\frac{z^n}{n^2}| = \frac{1}{n^2}$ so $\sum n^{-2} z^n$ is absolutely convergent

Using the ratio test: $|\frac{z^{n+1}}{(n+1)^2} / \frac{z^n}{n^2}| = \frac{|z|}{(1 + \frac{1}{n})^2} \rightarrow |z|$ so the radius of convergence is 1, and we have convergence on the circle of convergence

ii) $\sum_0^N z^n = \frac{1-z^{N+1}}{1-z}$ for $z \neq 1$ which tends to a limit iff $|z| < 1$

and $\sum_0^N1^n = N \rightarrow \infty$ so $\sum z^n$ has radius of convergence 1 and diverges at every point on the circle of convergence
\end{proof}

A more complicated example is given in question 16 on the 3rd example sheet.

It is a remarkable fact that we can operate with power series in the same way as polynomials (within the radius of convergence). In particular we shall show that we can differentiate term by term. 

\begin{AYTheorem} If $\sum_{n=0}^\infty a_n z^n $ has radius of convergence $R$ and we write $f(z)= \sum_{n=0}^\infty a_n z^n$ then $f$ is differentiable at all points $z$ with $|z| < R$ and 
\begin{equation*}
f'(z) = \sum_{n=1}^\infty n a_n z^{n-1}
\end{equation*}
\end{AYTheorem}

The proof is starred in the syllabus. We use three simple observations.

\begin{AYLemma}
If $\sum_{n=0}^\infty a_n z^n$ has radius of convergence $R$ then given any $\epsilon>0$ we can find a $K(\epsilon)$ such that $\sum_{n=0}^\infty |a_n z^n | < K(\epsilon)$ for all $|z| \leq R - \epsilon$.
\end{AYLemma}

\begin{AYLemma}
If $\sum_{n=0}^\infty a_n z^n $ has radius of convergence $R$ then so do $\sum_{n=1}^\infty n a_n z^{n-1}$ and $\sum_{n=2}^\infty n (n-1) a_n z^{n-2}$.
\end{AYLemma}

\begin{proof}
Suppose $\sum a_n z^n$ has radius of convergence $R$. If $|z| < R$ we can find $w_0$ with $|z| < |w_0| < R$

Since $|w_0| < R, \sum a_n w_0^n$ converges, so $\exists M$ such that $|a_n w_0^n| \leq M \ \forall n$

Now $|n a_n z^{n-1}| = |a_n w_0^n| |\frac{z}{w_0}|^{n-1} \frac{n}{|w_0|} \leq \frac{Mn}{w_0} |\frac{z}{w_0}|^{n-1} = u_n$

Claim that $\sum u_n$ converges $\because \frac{u_{n+1}}{u_n} = \frac{z}{w_0} (1 + \frac{1}{n}) \rightarrow \frac{z}{w_0} < 1$ so ratio test works.

Since $\sum u_n$ converges the comparison test tells us that $\sum |na_n z^{n-1}|$ converges, so $\sum na_n z^{n-1}$ converges.

Thus $\sum n a_n z^{n-1}$ has radius of convergence at least $R$, since $|n a_n z^{n-1}| \geq n |a_n| |z|^{n-1} = \frac{1}{|z|} n |a_n z^n|$

$\sum n (n-1) a_n z^{n-2}$ has radius of convergence $R$ by ``differentiating'' again.
\end{proof}

\begin{AYLemma}
(i) ${\displaystyle \binom{n}{r} \leq n(n-1) \binom{n-2}{r-2}}$ for all $2\leq r\leq n$.

(ii) $|(z+h)^n -z^n -n h z^{n-1}| \leq n(n-1)(|z| + |h|)^{n-2}|h|^{2}$ for all $z, h \in \mathbb{C}$.
\end{AYLemma}

\begin{proof}
i)
\begin{equation*}
n (n-1) \binom{n-2}{r-2} = \frac{n(n-1)(n-2)}{(r-2)!(n-r)!} = \frac{n!}{(r-2)!(n-r)!} \geq \frac{n!}{r!(n-r)!} = \binom{n}{r}
\end{equation*}

ii) 
\begin{align*}
|(z + h)^n - nz^{n-1} h - z^n| &= \sum_{r=2}^n |\binom{n}{r} z^{n-r} h^2|\\
&\leq n(n-1) \sum_{r=2}^n \binom{n-2}{r-2} |z^{n-r} h^r|\\
&= n(n-1) h^2 \sum_{r=0}^n \binom{n-2}{r} |z^{n-2-r}||h^r|\\
&= n(n-1)|h|^2 (|z| + |h|)^{n-2}
\end{align*}
Thus if $|z| + |h| < R$
\begin{align*}
|\frac{\sum a_n (z+h)^n - \sum a_n z^n}{h} - \sum n a_n z^{n-1}| &= |\frac{1}{h} (\sum a_n ((z+h)^n - z^n - nz^{n-1}h))|\\
&\leq \frac{1}{|h|} \sum |a_n| |(z+h)^n - z^n - nz^{n-1}h|\\
&\leq \frac{1}{|h|} \sum |a_n||h|^2 (|z| + |h|)^{n-2} n (n-1)\\
&= |h| \sum n (n-1) |a_n| (|z| + |h|)^{n-2}
\end{align*}
and given that we are within the radius of convergence this sum converges to a bounded value, so $\sum n (n-1) |a_n| |w|^n \leq K$ for $|w| < R - \epsilon$
\end{proof}

In this course we shall mainly work on the real line. Restricting to the real line we obtain the following result.

\begin{AYTheorem}
(i) If $a_j \in \mathbb{R}$ there exists a unique $R$ (the radius of convergence) with $0 \leq R \leq \infty$ such that $\sum_{n=0}^\infty a_n x^n $ converges for all real $x$ with $|x| < R$ and diverges for all real $x$ with $x > R$.

(ii) If $\sum_{n=1}^\infty a_n x^n $ has radius of convergence $R$ and we write $f(x)= \sum_{n=0}^\infty a_nx^n $ then $f$ is differentiable at all points $x$ with $|x|<R$ and 
\begin{equation*}
f'(x)=\sum_{n=1}^\infty na_nx^{n-1}
\end{equation*}
\end{AYTheorem}

\section{The standard functions}

In school you learned all about the functions $\exp$, $\log$, $\sin$ and $\cos$ and about the behaviour of $x^{\alpha}$. Nothing that you learned was wrong (we hope) but you might be hard pressed to prove all the facts you know in a coherent manner,

To get round this problem, we start from scratch making new definitions and assuming nothing about these various functions. One of your tasks is to make sure that the lecturer does not slip in some unproved fact. On the other hand you must allow your lecturer to choose definitions which allow an easy development of the subject rather than those that follow some `historic', `intuitive'  or `pedagogically appropriate' path\footnote{If you want to see a treatment along these lines see the excellent text of Burn\cite{Burn}.}.

Let us start with the exponential function. Throughout this section we shall restrict ourselves to the real line.

\begin{AYLemma}
The sum $\sum_{n=0}^\infty \frac{x^n }{n!}$ has infinite radius of convergence.
\end{AYLemma}

\begin{proof}
\begin{equation*}
|\frac{x^{n+1}}{(n+1)!}| / |\frac{x^n}{n!}| = \frac{|x|}{n+1} \rightarrow 0
\end{equation*}
so by the ratio test $\sum_{n=0}^\infty \frac{x^n}{n!}$ converges $\forall x$ so the radius of convergence is $\infty$
\end{proof}

We can thus define a function $e:\mathbb{R}\rightarrow\mathbb{R}$ by
\begin{equation*}
e(x)=\sum_{n=0}^\infty\frac{x^n }{n!}
\end{equation*}
(We use $e(x)$ rather than $\exp(x)$ to help us avoid making unjustified assumptions.)

\begin{AYTheorem}
\label{ExponentialRealProperties}
(i) The function $e : \mathbb{R} \rightarrow \mathbb{R}$ is everywhere differentiable with $e'(x) = e(x)$.

(ii) $e(x + y) = e(x)e(y)$ for all $x, y \in \mathbb{R}$. 

(iii) $e(x) > 0$ for all $x \in \mathbb{R}$.

(iv) $e$ is a strictly increasing function.

(v) $e(x) \rightarrow \infty$ as $x \rightarrow \infty$, $e(x) \rightarrow 0$ as $x \rightarrow -\infty$.

(vi) $e : \mathbb{R} \rightarrow (0,\infty)$ is a bijection.
\end{AYTheorem}

\begin{proof}
i) Since a power series can be differentiated term by term within its radius of convergence $e(x)$ is differentiable
\begin{equation*}
\therefore e'(x) = \sum_1^\infty n . \frac{x^{n-1}}{n!} = \sum_1^\infty \frac{x^{n-1}}{(n-1)!} = \sum_0^\infty \frac{x^n}{n!} = e(x)
\end{equation*}

ii) Let $f(x) = e(a-x) e(x)$ with $a$ a constant
\begin{align*}
f'(x) &= -e'(a-x) e(x) + e(a-x) e'(x)\\
&=-e(a-x)e(x) + e(a-x)e(x)\\
&= 0
\end{align*}
so by the mean value theorem, $f$ is constant, $\therefore f(x) = f(0)$ i.e. $e(a-x)e(x) = e(a) e(0)$

Looking at power series $e(0) = 1 + \frac{0}{1!} + \frac{0}{2!} + \cdots = 1$ so $e(a-x)e(x) = e(a) \ \forall a, x$

Set $a = x+y$ to get $e(y) e(x) = e(x + y)$

iii) $e(x) = \sum_0^\infty \frac{x^r}{r!} \geq 1 \ \forall x \geq 0$ but $(e^{-x}) e^x = e(-x + x) = e(0) = 1$

Therefore $e^{-x} > 0$ for $x > 0$, and thus $e^x > 0 \ \forall x$

iv) One of the corollaries of the mean value theorem (theorem \ref{MeanValueTheorem}) tells us that since $e'(x) = e(x) >0$ $e$ is strictly increasing

v) $e(x) = \sum_0^\infty \frac{x^r}{r!} \geq 1 + \frac{x}{1!} = 1+x \rightarrow \infty$ so $e(x) \rightarrow \infty$ as $x \rightarrow \infty$

$e(-x) = \frac{1}{e(x)}$ (by $e(-x) e(x) = e(0) = 1$), and $\frac{1}{e(x)} \rightarrow 0$ as $x \rightarrow \infty$, so $e(x) \rightarrow 0$ as $x \rightarrow -\infty$

vi) Since $e$ is strictly increasing it is injective ($x > y \Rightarrow e(x) > e(y)$)

If $0 < c$ since $e(x) \rightarrow 0$ as $x \rightarrow -\infty$ we can find $x_1$ with $e(x_1) < c$

Since $e(x) \rightarrow \infty$ as $x \rightarrow \infty$ we can find $x_3$ with $e(x_3) > c$

So by the Intermediate Value Theorem $\exists x_2$ such that $e(x_2) = c$

Thus $e$ is surjective
\end{proof}

It is worth stating some of our results in the language of group theory.

\begin{AYLemma}
The mapping $e$ is an isomorphism of the group $(\mathbb{R},+)$ and the group $((0,\infty),\times)$.
\end{AYLemma}

\begin{proof}
$e(x + y) = e(x) e(y)$
\end{proof}

Since $e : \mathbb{R} \rightarrow (0,\infty)$ is a bijection we can consider the inverse function $l : (0,\infty) \rightarrow \mathbb{R}$.

\begin{AYTheorem} 
(i) $l : (0,\infty) \rightarrow \mathbb{R}$ is a bijection. We have $l(e(x)) = x$ for all $x \in \mathbb{R}$ and $e(l(t)) = t$ for all $t \in (0,\infty)$.

(ii) The function $l:(0,\infty) \rightarrow \mathbb{R}$ is everywhere differentiable with $l'(t)=1/t$.

(iii) $l(uv) = l(u) + l(v)$ for all $u, v \in(0,\infty)$.
\end{AYTheorem}

\begin{proof}
i) Just repeats what it means to be an inverse function

ii) We know that the inverse of a bijective function $f$ is differentiable and $(f^{-1})'(x) = \frac{1}{f'(f^{-1}(x))}$

so $l'(x) = \frac{1}{e'(l(x))} = \frac{1}{e(l(x))} = \frac{1}{x}$

iii) Just isomorphism

$e$ goes from $(\mathbb{R}, +)$ to $((0, \infty), \times)$

$l$ goes from $((0, \infty), \times)$ to $(\mathbb{R}, +)$

(Or, $e(l(u) + l(v)) = e(l(u))e(l(v)) = uv = e(l(uv))$, so as $e$ is bijective, $l(u) + l(v) = l(uv)$)
\end{proof} 

We now write $e(x) = \exp(x)$, $l(t) = \log t$ (the use of $\ln$ is unnecessary). If $\alpha \in \mathbb{R}$ and $x>0$ we write $r_{\alpha}(x) = \exp(\alpha \log x)$.

\begin{AYTheorem}
\label{IndicesProperties}
Suppose $x,\, y > 0$ and $\alpha, \beta \in \mathbb{R}$. Then

(i) $r_{\alpha}(xy) = r_{\alpha}(x) r_{\alpha}(y)$,

(ii) $r_{\alpha + \beta}(x) = r_{\alpha}(x)r_{\beta}(x)$,

(iii) $r_{\alpha} (r_{\beta}(x)) = r_{\alpha\beta}(x)$

(iv) $r_1(x)=x$.
\end{AYTheorem}

\begin{proof}
i) 
\begin{align*}
r_\alpha(xy) &= \exp(\alpha \log(xy))\\
&= \exp(\alpha \log x + \alpha \log y)\\
&= \exp (\alpha \log x) \exp (\alpha log y)\\
&= r_\alpha(x) r_\alpha(y)
\end{align*}
ii)
\begin{align*}
r_{\alpha + \beta}(x) &= \exp ((\alpha + \beta)\log x)\\
&= \exp(\alpha \log x + \beta \log x)\\
&= \exp(\alpha \log x)\exp(\beta \log x)\\
&= r_\alpha(x) r_\beta(x)
\end{align*}
iii)
\begin{align*}
r_\alpha(r_\beta(x)) &= \exp(\alpha \log(\exp(\beta(\log x))))\\
&= \exp(\alpha \beta \log x)\\
&= r_{\alpha \beta}(x)
\end{align*}
iv)
\begin{align*}
r_1(x) = \exp(\log(x)) = x
\end{align*}
\end{proof}

\begin{AYExercise}
Use the results of Theorem~\ref{IndicesProperties} to show that if $n$ is a strictly positive integer and $x>0$ then
\begin{equation*}
r_n(x)=\underbrace{xx\dots x}_n
\end{equation*}
\end{AYExercise}

\begin{proof}
$r_n(x) = r_{\underbrace{1 + 1 + 1 + \cdots + 1}_{n \text{ times}}}(x) = \underbrace{r_1(x) r_1(x) r_1(x) \ldots r_1(x)}_{n \text{ times}} = \underbrace{x . x . x . \ldots . x}_{n \text{ times}}$
\end{proof}

Thus, if we write $x^{\alpha} = r_{\alpha}(x)$, our new notation is \emph{consistent} with our old school notation when $\alpha$ is rational but gives, in addition, a valid definition when $\alpha$ is irrational. 

\begin{AYLemma}
(i) If $\alpha$ is real, $r_{\alpha} : (0,\infty) \rightarrow (0,\infty)$ is everywhere differentiable and $r_{\alpha}'(x) = \alpha r_{\alpha -1}(x)$.

(ii) If $x>0$ and we define $f_{x} (\alpha) = x^{\alpha}$ then $f_{x} : \mathbb{R} \rightarrow (0,\infty)$ is everywhere differentiable and $f_{x}'(\alpha) = \log x f_{x}(\alpha)$.
\end{AYLemma}

\begin{proof}
i)
\begin{align*}
\frac{d}{dx} r_\alpha(x) &= \frac{d}{dx} (\exp(\alpha \log x))\\
&= \frac{d}{dx} (e(\alpha l(x)))\\
&= \alpha l'(x) e'(\alpha l(x)) \text{ (chain rule)}\\
&= \frac{\alpha}{x} e(\alpha l(x))\\
&= \frac{\alpha}{x} r_\alpha(x)\\
&= \alpha_{r-1} (x) r_\alpha(x)\\
&= \alpha r_{\alpha-1}(x)
\end{align*}

ii) $f_x(\alpha) = \exp(x \log \alpha)$

$\therefore (f_x)'(\alpha) = \log \alpha \exp'(x \log \alpha) = \log \alpha f_x (\alpha)$
\end{proof}

Note: This may appear circular, as we use $x^n$ in the definition of $e^x$, and then $e^x$ for $x^n$. However, the logic actually goes as follows:
\begin{quote}
i) Define $x^n$ for $n \in \mathbb{N}$, find properties

ii) Define $r_\alpha(x)$ for $\alpha \in \mathbb{R}, x > 0$, show that $r_n(x) = x^n$ for $n \in \mathbb{N}$

iii) Decide to write $r_alpha(x) = x^\alpha$
\end{quote}

Finally we look at the trigonometric functions.

\begin{AYLemma}
The sums $\sum_{n=0}^\infty \frac{(-1)^n x^{2n+1}}{(2n+1)!}$ and $\sum_{n=0}^\infty\frac{(-1)^n x^{2n}}{(2n)!}$ have infinite radius of convergence.
\end{AYLemma}

\begin{proof}
Observe that
\begin{equation*}
|\frac{(-1)^{n+1} x^{2n+3}}{(2n+3)!}| / |\frac{(-1)^n x^{2n+1}}{(2n+1)!}| = \frac{x^2}{(2n+2)(2n+2)} \rightarrow 0 \text{ as }n \rightarrow 0 \ \forall x
\end{equation*}

$\therefore \sum_{n=0}^\infty \frac{(-1)^n x^{2n+1}}{(2n+1)!}$ converges $\forall x$ i.e. infinite radius of convergence

Similarly for the other half
\end{proof}

We can thus define functions $s,\, c:\mathbb{R}\rightarrow\mathbb{R}$ by
\begin{equation*}
s(x)=\sum_{n=0}^\infty\frac{(-1)^n x^{2n+1}}{(2n+1)!} \text{ and }c(x)=\sum_{n=0}^\infty\frac{(-1)^n x^{2n}}{(2n)!}
\end{equation*}

\begin{AYLemma}
\label{TrigonometricFunctionsProperties}
(i) The functions $s,\, c : \mathbb{R} \rightarrow \mathbb{R}$ are everywhere differentiable with $s'(x) = c(x)$ and $c'(x) = -s(x)$.

(ii) $s(x + y) = s(x)c(y) + c(x)s(y)$, $c(x + y)=c(x)c(y) - s(x)s(y)$ and $s(x)^{2} + c(x)^{2}=1$ for all $x, y \in \mathbb{R}$.

(iii) $s(-x) = -s(x)$, $c(-x) = c(x)$ for all $x \in \mathbb{R}$.
\end{AYLemma}

\begin{proof}
i) We can differentiate term by term within the radius of convergence so, for example
\begin{equation*}
c'(x) = \sum_{n=1}^\infty \frac{d}{dx} (\frac{x^{2n} (-1)^n}{(2n)!}) = \sum_1^\infty (-1) \frac{x^{2n-1}(-1)^{n-1}}{(2n-1)!} = -\sum_0^\infty \frac{(-1)^n x^{2n+1}}{(2n+1)!} = -s(x)
\end{equation*}

ii) Fix $\alpha, \beta$, let $a = \alpha + \beta$, Consider
\begin{align*}
f(x) &= s(x) c(a-x) + s(a-x)c(x)\\
f'(x) &= s'(x) c(a-x) - s(x) c'(a-x) - s'(a-x) c(x) + s(a-x) c'(x)\\
&= c(x) c(a-x) - s(x) s(a-x) - c(a-x) c(x) - s(a-x) s(x) \\
&= 0
\end{align*}
sp $f$ is constant, so $f(x) = f(a) \ \forall x$, i.e. $s(x) c(a-x) + s(a-x)c(x) = s(a) c(a-a) + s(a-a)c(a) = s(a)c(0) + s(0)c(a)$

From the power series $c(0) = 1, s(0) = 0$, so not set $x = \beta$ to get
\begin{equation*}
s(\beta)c(\alpha) + s(\alpha)c(\beta) = s(\alpha + \beta)
\end{equation*}

$c(x+y)$ is proved similarly

\begin{align*}
\frac{d}{dx}(c^2(x) + s^2(x)) &= 2c'(x) c(x) + 2s'(x) s(x)\\
&= -2s(x)c(x) + 2s(x) c(x)\\
&= 0
\end{align*}

$\therefore c^2(x) + s^2(x)$ is constant, and thus $c^2(x) + s^2(x) = c^2(0) + s^2(0) = 1 + 0 = 1$ and thus $c^2(x) + s^2(x) = 1 \ \forall x$

iii) $s(-x) = \sim_{n=0}^\infty \frac{(-1)^n (-x)^{2n+1}}{(2n+1)!} = \sum_{n=0}^\infty - \frac{(-1)^n x^{2n+1}}{(2n+1)!} = -s(x)$

Similarly $c(-x) = c(x)$
\end{proof}

\begin{AYExercise}
Write down what you consider to be the chief properties of $\sinh$ and $\cosh$. (They should convey enough information to draw a reasonable graph of the two functions.)

(i) Obtain those properties by starting with the definitions 
\begin{equation*}
\sinh x = \sum_{n=0}^\infty \frac{x^{2n+1}}{(2n+1)!} \text{ and }\cosh x = \sum_{n=0}^\infty \frac{x^{2n}}{(2n)!}
\end{equation*}
and proceeding along the lines of Lemma~\ref{TrigonometricFunctionsProperties}.

(ii) Obtain those properties by starting with the definitions
\begin{equation*}
\sinh x = \frac{\exp x-\exp (-x)}{2} \text{ and }\cosh x = \frac{\exp x+\exp (-x)}{2}
\end{equation*}
\end{AYExercise}

We have not yet proved one of the most remarkable properties of the sine and cosine functions, their periodicity.

\begin{AYTheorem}
Let $s$ and $c$ be as in Lemma~\ref{TrigonometricFunctionsProperties}.

(i) If $c(a)=0$ and $c(b) = 0$ then $s(b-a)=0$.

(ii) We have $s(x) > 0$ for all $0 < x \leq 1$.

(iii) There exists a unique $\omega \in [0,2]$ such that $c(\omega)=0$.

(iv) $s(\omega) = 1$.

(v) $s(x + 4\omega) = s(x)$, $c(x + 4 \omega) = c(\omega)$.

(vi) The function $c$ is strictly decreasing from $1$ to $-1$ as $x$ runs from $0$ to $2\omega$,
\end{AYTheorem}

\begin{proof}
i)
\begin{align*}
s(b-a) &= s(b)c(-a) + s(-a)c(b)\\
&=s(b)c(a) - s(a)c(b)\\
&= 0 - 0 = 0
\end{align*}

ii) $s(x)$ by $s(x) = x - \frac{x^3}{3!} + \frac{x^5}{5!} - \cdots$

and if $0 < x \leq 1$, $\frac{x^{2n+1}}{(2n+1)!} < \frac{x^{2n-1}}{(2n-1)!}$ (proof: $\frac{x^{2n+1}}{(2n+1)!} / \frac{x^{2n-1}}{(2n-1)!} = \frac{x^2}{(2n+1)(2n)} < 1$)

The error in computing an alternating [convergent] series is less than the first term omitted, so $|s(x) - x| \leq \frac{x^3}{3!}$ so $s(x) \geq x - \frac{x^3}{3!} > 0$ for $x \in (0, 1]$

So by i) if $a, b$ are distinct zeroes of $c(x)$ we have $|a-b|>1 \qquad (\dagger)$

iii) If $0 \leq x \leq 1$ then the terms $\frac{x^{2n}}{(2n)!}$ are strictly decreasing, tending to 0

So by alternating series the error in
\begin{equation*}
c(x) = \sum (-1)^n \frac{x^{2n}}{(2n)!}
\end{equation*}
is equal to the first term neglected, so $|c(x) - 1| \leq \frac{x^2}{2!}$ so $c(x) \geq 1 - \frac{x^2}{2!} \geq \frac{1}{2} \ \forall x \in [0, 1]$

Look at $c(2)$: $c(2) = 1 - \frac{2^2}{2!} + \frac{2^4}{4!} - \frac{2^6}{6!} + \cdots$, not decreasing, alternating, but $-\frac{2^2}{2!} + \frac{2^4}{4!} - \frac{2^6}{6!}$ is

So $|c(2) - (1-\frac{2^2}{2!})| \leq \frac{2^4}{4!}$, sp $|c(2) + 1| \leq \frac{2}{3}$, so $c(2) \leq -\frac{1}{3}$

By the Intermediate Value Theorem since $c(1) > 0 > c(2) \ \exists \omega \in [1, 2]$ such that $c(\omega) = 0$. By the earlier result $(\dagger)$ it is unique.

The rest is simple algebra, using
\begin{align*}
c(2\omega)&=c(\omega)c(\omega) - s(\omega)s(\omega)\\
&=-s^2(\omega)\\
&=-1 + c^2(\omega)\\
&=-1
c(4\omega) &= c(2\omega)c(2\omega) - s(2\omega)s(2\omega)\\
&=1 + (1-c^2(2\omega))\\
&=1
\end{align*}

$s'(x) = c(x) \geq 0$ for $0 \leq x \leq \omega$, so since $s(0) =0$ the Mean Value Theorem says $s(x) \geq 0$ for $0 \leq x \leq \omega$

$s^2(\omega) + c^2(\omega) = 1$ so $s^2(\omega) = 1$ so since $s(\omega) \geq 0, s(\omega) = 1$

$s(2\omega) = 2s(\omega)c(\omega) = 0$ and $s(4\omega) = 2s(2\omega)c(2\omega) = 0$

$s(x+4\omega) = s(x) c(4\omega) + c(x) s(4\omega) = s(x)$ and $c(x+4\omega) = c(x) c(4\omega) - s(x) s(4\omega) = c(x)$
\end{proof}

We now {\bf define} $\pi = 2\omega$. If $\overline{x}$ and $\overline{y}$ are non-zero vectors in $\mathbb{R}^{m}$ we know by the Cauchy-Schwarz inequality that $|\overline{x}.\overline{y}|\leq \|\overline{x}\|\|\overline{y}\|$ and we may {\bf define} the angle between the two vectors to be that $\theta$ with $0 \leq \theta \leq \pi$ which satisfies
\begin{equation*}
\cos \theta=\frac{\overline{x}.\overline{y}}{\|\overline{x}\|\|\overline{y}\|}
\end{equation*}

We can also justify the standard use of polar coordinates.
 
\begin{AYLemma}
If $(x,y) \in \mathbb{R}^{2}$ and $(x,y) \neq(0,0)$ then there exist a unique $r > 0$ and $\theta$ with $0 \leq \theta < 2 \pi$ such that
\begin{equation*}
x = r \cos \theta, y = r \sin \theta
\end{equation*}
\end{AYLemma}

\begin{proof}
If $\underline{x}, \underline{y}$ are non-zero then Caughy Schwarz tells us
\begin{equation*}
-1 \leq \frac{\underline{x} . \underline{y}}{||\underline{x}|| \ ||\underline{y}||} < 1
\end{equation*}
We have already seen that $\cos ' \theta = - \sin \theta < 0$ for $0 < \theta < \pi$ so $\cos$ is strictly decreasing, and continous on $[0, \pi]$ with $\cos 0 = 1, \cos \pi = -1$ so the equation
\begin{equation*}
\cos \theta = \frac{\underline{x} . \underline{y}}{||\underline{x}|| \ ||\underline{y}||}
\end{equation*}
has exactly only solution $\theta$ with $0 \leq \theta \leq \pi$. We call the angle between $\underline{x}$ and $\underline{y}$ $\theta$

Suppose that $(x, y) \neq (0, 0)$, then
\begin{equation*}
-1 \leq \frac{x}{\sqrt{x^2 + y^2}} \leq 1
\end{equation*}
so as before there is $\theta_1 \in [0, \pi]$ such that $\cos \theta_1 = \frac{x}{\sqrt{x^2 + y^2}}$ and there is also $\theta_2 \in [\pi, 2\pi]$ such that $\cos \theta_2 = \frac{x}{\sqrt{x^2 + y^2}}$

$\cos^2 \theta_1 + \sin^2 \theta_1 = 1 = \cos^2 \theta_2 + \sin^2 \theta_2$, $\frac{x}{\sqrt{x^2 + y^2}} + \frac{y}{\sqrt{x^2 + y^2}} = 1$ and $\sin \theta_1 \geq 0, \sin \theta_2 \leq 0$ so we can choose a unique $\theta \in [0, 2 \pi]$ such that $\cos \theta = \frac{x}{\sqrt{x^2 + y^2}}, \sin \theta = \frac{y}{\sqrt{x^2 + y^2}}$

(we must take care if $(x, y) = (1,0)$ or $(-1,0)$)

Writing $r$ for the positive square root of $(x^2 + y^2)$

\begin{equation*}
x = r \cos \theta \qquad y = r \sin \theta
\end{equation*}
\end{proof}

You may be unhappy with a procedure which reduces geometry to analysis. It is possible to produce treatments which soften the blow\footnote{The matter is more subtle than it looks. Classical Euclidean geometry is `weaker' than the geometry required to justify analysis and if you wished to obtain analysis from geometry you would need to add extra axioms.} but what we can not do is to justify analytic results by appealing to geometry and then appeal to analysis to justify the geometry.

\section{Onwards to the complex plane}

This section contains useful background material which does not really form part of the course. If time is short I shall omit it entirely.

The mean value theorem {\bf fails} for differentiable functions $f : \mathbb{C} \rightarrow \mathbb{C}$. (See Example~\ref{NoComplexMVT}.) However, the constant value theorem holds.

\begin{AYTheorem}
\label{ComplexZeroDerivativeConstant}
If $f : \mathbb{C} \rightarrow \mathbb{C}$ is differentiable and $f'(z) = 0$ for all $z \in \mathbb{C}$ then $f$ is constant.
\end{AYTheorem}

\begin{proof}
(This is an ad hoc proof, better ones will follow)

Fix $y_0$, we show that $\Re(f(x + iy_0))$ is constant
\begin{quote}
\begin{proof}
Set $u(x) = \Re(f(x + iy_0))$
\begin{align*}
\frac{u(x + \delta x)- u(x)}{\delta x} &= \Re(\frac{f(x + \delta x + i y_0) - f(x + iy_0)}{\delta x})\\
&=\Re(f'(x + i y_0))\\
&= 0
\end{align*}
So $u$ is differentiable and $u'(x) = 0 \ \forall x$, and $u: \mathbb{R} \rightarrow \mathbb{R}$ so $u$ is constant

Now set $v(y) = \Re(f(x_0 + i y_0))$ ($x_0$ fixed)
\begin{align*}
\frac{v(y + \delta y) - v(y)}{\delta y} &= \Re(\frac{f(x_0 + iy + i \delta y) - f(x_0 + iy)}{\delta y})\\
&= \Re(i.\frac{f(x_0 + iy + i \delta y) - f(x_0 + iy)}{i \delta y})\\
&= \Re(i f'(x_0 + iy))\\
&= 0
\end{align*}
so $v$ is constant, and as $\Re(f(x + iy))$ is constant if we vary $x$ and $y$ $\Re(f(x_1 + i y_1)) = \Re(f(x_1 + i y_2)) = \Re(f(x_2 + i y_2))$ so $\Re(f(x))$ is constant
\end{proof}
\end{quote}
Similarly $\Im(f(x))$ is constant (or look at $\Re(if(x))$), so $f$ is constant
\end{proof}

The proof of Theorem~\ref{ComplexZeroDerivativeConstant} that I shall give is very ad hoc and you will meet better ones later.

Since the constant value theorem holds we can extend the proof of Theorem~\ref{ExponentialRealProperties}~(ii) to this wider context and obtain a version of the exponential function for complex numbers. In this section we work in $\mathbb{C}$ unless otherwise stated.

\begin{AYLemma}
The sum $\sum_{n=0}^\infty \frac{z^n }{n!}$ has infinite radius of convergence.
\end{AYLemma}

\begin{proof}
\begin{equation*}
|\frac{z^{n+1}}{(n+1)!} / \frac{z^n}{n!}| = \frac{|z|}{n+1} \rightarrow 0 \text{ as }n \rightarrow \infty
\end{equation*}
\end{proof}

We can thus define a function $\exp : \mathbb{C} \rightarrow \mathbb{C}$ by 
\begin{equation*}
\exp(z)=\sum_{n=0}^\infty\frac{z^n }{n!}
\end{equation*}

\begin{AYTheorem}
(i) The function $e : \mathbb{C} \rightarrow \mathbb{C}$ is everywhere differentiable with $e'(z) = e(z)$.

(ii) $e(z + w) = e(z) e(w)$ for all $z, w \in \mathbb{C}$.
\end{AYTheorem}

\begin{proof}
i) Differentiate term by term

ii) Show that $\frac{d}{dz}(e(a-z)e(z)) = 0$ for $a$ constant

Use the constant value theorem to deduce that $e(a-z)e(z) = e(a) \ \forall z$ so $e(z-z)e(z) = e(a) \ \forall z, \forall a$ and so $e(w)e(z) = e(z+w)$
\end{proof}

Notice that the remaining parts of Theorem~\ref{ExponentialRealProperties} are either meaningless or false (compare Theorem~\ref{ExponentialRealProperties}~(vi) with Lemma~\ref{ExponentialComplexProperties}~(iii) which shows that $\exp:\mathbb{C}\rightarrow\mathbb{C}$ is not injective). We must be very careful in making the transition from real to complex.

We obtain a series of famous formulae.

\begin{AYLemma}
\label{ExponentialComplexProperties}
(i) If $\theta$ is real 
\begin{equation*}
\exp i \theta = \cos \theta + i \sin \theta
\end{equation*}

(ii) If $x$ and $y$ are real
\begin{equation*}
\exp (x + iy) = (\exp x) (\cos y + i \sin y)
\end{equation*}

(iii) $\exp$ is periodic with period $2 \pi i$, that is to say
\begin{equation*}
\exp(z + 2 \pi i) = \exp z
\end{equation*}
for all $z \in \mathbb{C}$.

(iv) $\exp(\mathbb{C}) = \mathbb{C} \setminus \{ 0 \}$.
\end{AYLemma}

\begin{proof}
i)
\begin{align*}
\exp(i \theta) = \sum_{n=0}^\infty \frac{i \theta}{n!} &= \sum_{n=0}^\infty \frac{(i \theta)^{2n}}{(2n)!} + \sum_{n=0}^\infty \frac{(i \theta)^{2n+1}}{(2n+1)!}\\
&= \sum_{n=0}^\infty \frac{(-1)^n \theta^{2n}}{(2n)!} + i \sum_{n=0}^\infty \frac{(-1)^n \theta^{2n+1}}{(2n+1)!}\\
&= \cos \theta + i \sin \theta \text{ for }\theta \in \mathbb{R}
\end{align*}
ii)
\begin{align*}
\exp(x + iy) &= \exp(x) \exp (iy)\\
&= \exp(x) (\cos y + i \sin y) \text{ for }x, y \in \mathbb{R}
\end{align*}
iii)
\begin{align*}
\exp(z + 2\pi i) &= \exp(z) \exp(2 \pi i)\\
&= \exp(z) (\cos 2 \pi + i \sin 2\pi)\\
&= \exp z
\end{align*}
so $\exp$ is periodic with period $2 \pi i$
iv) If $z \in \mathbb{C}$ and $z \neq 0$ then $z = x + iy$ with $(x, y) \neq (0,0)$, so $z = r \cos \theta + i r \sin \theta$ for some $r > 0, \theta \in \mathbb{R}$, which in turn is equal to $r \exp(i \theta) = \exp(\log r) \exp (i \theta) = \exp(\log r + i \theta)$ so $\exp(\mathbb{C}) \supset \mathbb{C} \setminus \{0\}$

It is impossible to have $\exp(\omega) = 0$ for then $\exp(z) = \exp(z - \omega) \exp(\omega) = 0 \ \forall z$, but set $z = 0, \exp(0) = 1 \#$

So the range of $\exp$ is $\mathbb{C} \setminus \{0\}$, and thus $\exp$ takes every value except 0 infinitely often, as it is periodic
\end{proof}

\begin{AYExample}
\label{NoComplexMVT} 
Observe that $\exp 0 = \exp 2 \pi i = 1$ but $\exp'(z) = \exp(z) \neq 0$ for all $z \in \mathbb{C}$. Thus the  mean value theorem does not hold for differentiable functions $f : \mathbb{C} \rightarrow \mathbb{C}$.
\end{AYExample}

It is also possible to extend the definition of sine and cosine to the complex plane but the reader is warned that the behaviour of the new functions may be somewhat unexpected. Since these extended functions most certainly do not form part of this course (though you will be expected to know them after the Complex Methods course) their study is left as an exercise.

\begin{AYExercise}
(i) Explain why the infinite sums
\begin{equation*}
\sin z = \sum_{n=0}^\infty \frac{(-1)^n z^{2n+1}}{(2n+1)!} \text{ and } \cos z = \sum_{n=0}^\infty \frac{(-1)^n z^{2n}}{(2n)!}
\end{equation*}
converge everywhere and are differentiable everywhere with $\sin' z = \cos z$, $\cos' z = -\sin z$.

(ii) Show that
\begin{equation*}
\sin z = \frac{\exp iz-\exp (-iz)}{2i}, \cos z = \frac{\exp iz + \exp (-iz))}{2}
\end{equation*}
and
\begin{equation*}
\exp iz = \cos z + i\sin z
\end{equation*}
for all $z \in \mathbb{C}$.

(iii) Show that
\begin{equation*}
\sin (z + w) = \sin z\cos w + \cos z\sin w, \cos(z + w)=\cos z \cos w - \sin z\sin w
\end{equation*}
and $(\sin z)^{2} + (\cos z)^{2} = 1$ for all $z, w \in \mathbb{C}$.

(iv) $\sin(-z) = -\sin z$, $\cos (-z) = \cos z$ for all $z \in \mathbb{C}$.

(v) $\sin$ and $\cos$ are $2\pi$ periodic in the sense that
\begin{equation*}
\sin (z+2\pi)=\sin z \text{ and } \cos (z + 2\pi) = \cos z\] for all $z \in \mathbb{C}$.

(vi) If $x$ is real then $\sin ix = i\sinh x$ and $\cos ix = \cosh x$.

(vii) Recover the addition formulae for $\sinh$ and $\cosh$ by setting $z = ix$ and $w = iy$ in part~(iii).

(ix) Show that $|\sin z|$ and $|\cos z|$ are bounded if $|\Im z| \leq K$ for some $K$ but that $|\sin z|$ and $|\cos z|$ are unbounded on $\mathbb{C}$.
\end{AYExercise}

\begin{proof}
ii)
\begin{align*}
\exp(iz) - \exp(-iz) &= \sum_0^\infty \frac{(iz)^n}{n!} - \frac{(-iz)^n}{n!}\\
&= \sum_0^\infty \frac{1}{n!} (i^n - (-i)^n)z^n\\
&= \sum_0^\infty \frac{1}{(2r+1)!} 2 i^{2r+1} z^{2r+1}\\
&= 2i \sum_0^\infty \frac{1}{(2r+1)!} (-1)^r z^{2r+1}
\end{align*}
so $\sin z = \frac{exp(iz) - exp(-iz)}{2i}$ and similarly $\cos z = \frac{\exp(iz) + \exp(-iz)}{2}$

For the other parts

iii) Use ii) or i) and the CVT

iv) Use ii)

v) By iii) $\sin(z + 2 \pi) = \sin z \cos 2 \pi + \cos z \sin 2\pi = \sin z$

vi) Use ii)
ix) Use ii), $|z_1 - z_2| \leq \Big| |z_1| - |z_2| \Big|$
\end{proof}

However, as you were already shown in the Algebra and Geometry course and will be shown again in the Complex Methods course

\begin{center}
{\bf There is no logarithm defined on all of $\mathbb{C} \setminus \{ 0 \}$.}
\end{center}

\begin{AYExercise}
\label{NoComplexLogarithm}
Suppose, if possible, that there exists a continuous $L : \mathbb{C} \setminus \{ 0 \} \rightarrow \mathbb{C}$ with $\exp(L(z)) = z$ for all $z \in \mathbb{C} \setminus \{ 0 \}$.

(i) If $\theta$ is real, show that $L(\exp(i\theta)) = i(\theta + 2 \pi n(\theta))$ for some $n(\theta) \in \mathbb{Z}$.

(ii) Define $f:\mathbb{R}\rightarrow\mathbb{R}$ by
\begin{equation*}
f(\theta) = \frac{1}{2\pi} \left(\frac{L(\exp i\theta) - L(1)}{i} - \theta\right)
\end{equation*}
Show that $f$ is a well defined continuous function, that $f(\theta) \in {\mathbb Z}$ for all $\theta \in \mathbb{R}$, that $f(0) = 0$ and that $f(2\pi) = -1$.

(iii) Show that the statements made in the last sentence of (ii) are incompatible with the intermediate value theorem and deduce that no function can exist with the supposed properties of $L$.

(iv) Discuss informally what connection, if any, the discussion above has with the existence of the international date line.
\end{AYExercise}

\begin{proof}
Let us assume that such a $L$ exists

$\exp(L(\exp (i \theta))) = \exp(i \theta)$ so $L(\exp(i \theta)) = i \theta + 2 N \pi i$ for some integer $N$, call this $L(\exp(i \theta)) = i \theta + 2 \pi n(\theta)i$

Let $f(\theta) = \frac{1}{2 \pi} (\frac{L(\exp (i \theta)) - L(1)}{i} - \theta) = \frac{1}{2 \pi} (\frac{i \theta + 2 \pi  n(\theta) i - 2 \pi n(0) i}{i}) = n(\theta) - n(0) \in \mathbb{Z}$

$f$ is continuous (from chain rule etc), and $f(0) = \frac{1}{2\pi}(\frac{L(1) - L(1)}{i}) = 0, f(1) = \frac{1}{2 \pi}(\frac{L(1) - L(1)}{i} - 2 \pi) = -1$

Therefore the Intermediate Value Theorem states that $\exists c$ such that $f(c) = -\frac{1}{2} \notin \mathbb{Z}$ \#
\end{proof}

A similar argument shows that it is not possible to produce a continuous square root on the complex plane. 

\begin{AYExercise}
Show by modifyiing the argument of Exercise~\ref{NoComplexLogarithm}, that there does not exist a continuous $S : \mathbb{C} \rightarrow \mathbb{C}$ with $S(z)^{2} = z$ for all $z \in \mathbb{C}$.
\end{AYExercise}

More generally, it is not possible to define continuous non-integer powers $z^{\alpha}$ on the complex plane. (Of course, $z \mapsto z^n $ is well behaved if $n$ is an integer.) However, in the special case when $x$ is real and strictly positive we can define
\begin{equation*}
x^{z} = \exp(z\log x)
\end{equation*}
without problems and this enables us to write $\exp z = e^{z}$ where $e = \exp 1$. Surprisingly, Exercise~\ref{NoComplexLogarithm} is not an end but a beginning of much important mathematics -- but that is another story.

(The next bit was given in lectures, but did not form part of the printed notes)

Take the circle $\{e^{i \theta} : 0 \leq \theta < 2 \pi\}$. Say that $e^{i \theta_1} \sim e^{i \theta_2}$ iff $\theta_1 - \theta_2 = 2 \pi \mathbb{Z}$, so there are equivalence classes $[e^{i \phi}] = \{e^{i \theta} : e^{i \theta} \sim e^{i \phi}\}$

Pick one member from each equivalence class (one representative from each class) and call the result $E$

There are countably many rationals in $[0, 1)$, call them $x_1, x_2, x_3, \ldots$

Look at $E_n = \{e^{2\pi i x_n + i \theta} : e^{i \theta} \in E\} = e^{2 \pi i x_n}E$

$E_n \cap E_m = \phi$ and $\bigcup_{\forall n } E_n$ = whole circle

So what is the ``mass'' of $E_n$ if the circle ``weighs'' 1g?

We have a paradox - if it's posiitve then the circle weighs $\infty$ (made up of a union of countable infinite objects), and if 0 then the circle weighs 0

Therefore we have constructed a set with no ``natural'' definition of area. We need to carefully define area and also integrals as a result and even still we will find some integrals which cannot be found.

\section{The Riemann integral}

At school we are taught that an integral is the area under a curve. If pressed the framer of this definition might reply that
everybody knows what area is, but then everybody knows what honey tastes like. But does honey taste the same to you as it does to me? Perhaps the question is unanswerable but, for many practical purposes, it is sufficient that we agree on what we call honey.

In order to agree on what an integral is, we need a definition which does not depend on intuition. It is important that, as far as possible, the properties of our formally defined integral shall agree with our intuitive ideas on area {\bf but we have to prove this agreement} and not simply assume it.

In this section we introduce a notion of integral due to Riemann. For the moment we only attempt to define our integral for bounded functions on bounded intervals.

Let $f : [a,b] \rightarrow \mathbb{R}$ be a function such that there exists a $K$ with $|f(x)| \leq K$ for all $x \in [a,b]$. [To see the connection with `the area under the curve' it is helpful to suppose initially that $0 \leq f(x) \leq K$. However, all the definitions and proofs work more generally for  $-K \leq f(x) \leq K$.]

A dissection $\mathcal{D}$ of $[a,b]$ is a finite subset of $[a,b]$ containing the end points $a$ and $b$. By convention, we write
\begin{equation*}
\mathcal{D} = \{ x_{0}, x_1, \ldots, x_n\} \text{ with } a = x_{0} \leq x_1 \leq x_2 \leq \ldots \leq x_n = b
\end{equation*}
We define the \emph{upper sum} and \emph{lower sum}  associated with $\mathcal{D}$ by
\begin{align*}
S(f,\mathcal{D}) = & \sum_{j=1}^n (x_j-x_{j-1}) \sup_{x \in [x_{j-1},x_j]}f(x),\\
s(f,\mathcal{D}) = & \sum_{j=1}^n (x_j-x_{j-1}) \inf_{x \in [x_{j-1},x_j]}f(x)
\end{align*}
[Observe that, \emph{if the integral $\int_{a}^{b}f(t)\,dt$ exists,} then the upper sum ought to provide an upper bound and the lower sum a lower bound for that integral.]

\begin{figure}[htbp]
	\centering
		\includegraphics{Images/AY02.jpeg}
	\caption{Upper and Lower sums for integrals}
	\label{Upper and Lower sums for integrals}
\end{figure}

The next lemma is obvious but useful. 
\begin{AYLemma}
If $\mathcal{D}$ and $\mathcal{D}'$ are dissections with $\mathcal{D}' \supseteq \mathcal{D}$ then
\begin{equation*}
S(f,\mathcal{D})\geq S(f,\mathcal{D}') \geq s(f,\mathcal{D}')\geq s(f,\mathcal{D})
\end{equation*}
\end{AYLemma}

\begin{proof}
If $x_1 \geq y \geq x_0$ then
\begin{align*}
(x_1 - y) \sup_{t \in [y, x_1]} f(t) + (y - x_0) \sup_{t \in [y, x_0]} f(t) &\leq (x_1 - y) \sup_{t \in [x_0, x_1]} f(t) + (y-x) \sup_{t \in [x_0, x_1]} f(t)\\
&=(x_1 - x_0) \sup_{t \in [x_0, x_1]} f(t)
\end{align*}
So adding one point does not increase the upper sum, thus by induction
\begin{equation*}
S(f,\mathcal{D})\geq S(f,\mathcal{D}') \geq s(f,\mathcal{D}')\geq s(f,\mathcal{D})
\end{equation*}
\end{proof}

The next lemma is again hardly more than an observation but it is the key to the proper treatment of the integral. 

\begin{AYLemma}[Key integration property]
If $f : [a,b] \rightarrow \mathbb{R}$ is bounded and $\mathcal{D}_1$ and $\mathcal{D}_2$ are two dissections, then
\begin{equation*}
S(f,\mathcal{D}_1) \geq S(f,\mathcal{D}_1\cup\mathcal{D}_2) \geq s(f,\mathcal{D}_1\cup\mathcal{D}_2) \geq s(f,\mathcal{D}_2).
\tag*{$\bigstar$}
\end{equation*}
\end{AYLemma}

\begin{proof}
Immediate from the preceding lemma
\end{proof}

The inequalities $\bigstar$ tell us that, whatever dissection you pick and whatever dissection I pick, your lower sum cannot exceed my upper sum. There is no way we can put a quart in a pint pot\footnote{Or put a litre in a half litre bottle.}.

Since $S(f,\mathcal{D}) \geq -(b-a)K$ for all dissections $\mathcal{D}$  we can define the \emph{upper integral} as $I^{*}(f) = \inf_{\mathcal{D}}S(f,\mathcal{D})$. We define the \emph{lower integral} similarly as $I_{*}(f) = \sup_{\mathcal{D}}s(f,\mathcal{D})$. The inequalities $\bigstar$ tell us that these concepts behave well.

\begin{AYLemma}
If $f : [a,b] \rightarrow \mathbb{R}$ is bounded, then $I^{*}(f) \geq I_{*}(f)$.
\end{AYLemma}

\begin{proof}
$S(f, D_1) \geq s(D_2, f)$, thence immediate
\end{proof}

[Observe that, \emph{if the integral $\int_{a}^{b}f(t)\,dt$ exists,} then the upper integral ought to provide an upper bound and the lower integral a lower bound for that integral.]

If $I^{*}(f)=I_{*}(f)$, we say that $f$ is Riemann integrable and we write 
\begin{equation*}
\int_{a}^{b}f(x)\,dx=I^{*}(f)
\end{equation*}

The following lemma provides a convenient criterion for Riemann integrability.

\begin{AYLemma}
\label{RiemannIntegrableCriterion}
(i) A bounded function $f : [a,b] \rightarrow \mathbb{R}$ is Riemann integrable if and only if, given any $\epsilon > 0$, we can find a dissection $\mathcal{D}$ with
\begin{equation*}
S(f,\mathcal{D}) - s(f,\mathcal{D}) < \epsilon
\end{equation*}

(ii) A bounded function $f : [a,b] \rightarrow \mathbb{R}$ is Riemann integrable with integral $I$ if and only if, given any $\epsilon > 0$, we can find a dissection $\mathcal{D}$ with
\begin{equation*} 
S(f,\mathcal{D})-s(f,\mathcal{D})<\epsilon \ \text{and}\ |S(f,\mathcal{D})-I|\leq\epsilon
\end{equation*}
\end{AYLemma}

\begin{proof}
i) Suppose $f$ is Riemann integrable. Then $I^*(f) = I_*(f) = If$ say

Given any $\epsilon > 0$ we can find $\mathcal{D}_1$ such that $I^*(f) \geq S(f, \mathcal{D}_1) - \epsilon$ and $\mathcal{D}_2$ such that $I_*(f) \leq s(f, \mathcal{D}_2) + \epsilon$

\begin{align*}
\therefore I*(f) + \epsilon &\geq S(f, \mathcal{D}_1)\\
&\geq S(f, \mathcal{D}_1 \cup \mathcal{D}_2)\\
&\geq s(f, \mathcal{D}_1 \cup \mathcal{D}_2)\\
&\geq s(f, \mathcal{D}_2) = I_*(f) - \epsilon
\end{align*}

So $S(f, \mathcal{D}_1 \cup \mathcal{D}_2) - s(f, \mathcal{D}_1 \cup \mathcal{D}_2)< 2 \epsilon$

Conversely, suppose given any $\epsilon > 0$ we can find $\mathcal{D}$ with $S(f, \mathcal{D}) - s(f, \mathcal{D}) < \epsilon$. Then
\begin{align*}
0 &\leq I^*(f) - I_*(f)\\
&\leq S(f, \mathcal{D}) - s(f, \mathcal{D})\\
&< \epsilon
\end{align*}
So, since $\epsilon$ is arbitrary $I^*(f) = I_*(f)$

ii) Observe that i) shows the existence of the integral and note that
\begin{equation*}
S(f, \mathcal{D}) \geq I^*(f) \geq I_*(f) \geq s(f, \mathcal{D})
\end{equation*}
so if $S(f, \mathcal{D}) - s(f, \mathcal{D})< \epsilon$, $|S(f, \mathcal{D}) - I^*(f)| < \epsilon$ and $|s(f, \mathcal{D}) - I_*(f)| < \epsilon$
\end{proof}

Many students are tempted to use Lemma~\ref{RiemannIntegrableCriterion}~(ii) as the \emph{definition} of the Riemann integral.
The reader should reflect that, without the inequality $\bigstar$, it is not even clear that such a definition gives a unique value for $I$. (This is only the first of a series of nasty problems that arise if we attempt to develop the theory without first proving $\bigstar$, so I strongly advise the reader not to take this path.)

We now prove a series of standard results on the integral.

\begin{AYLemma}
(i) The function $J : [a,b] \rightarrow \mathbb{R}$ given by $J(t) = 1$ is integrable and
\begin{equation*}
\int_{a}^{b} 1\,dx=b-a
\end{equation*}

(ii) If $f, g : [a,b] \rightarrow \mathbb{R}$ are Riemann integrable, then so is $f + g$ and
\begin{equation*}
\int_{a}^{b} f(x) + g(x)\,dx = \int_{a}^{b}f(x)\,dx + \int_{a}^{b}g(x)\,dx
\end{equation*}

(iii) If $f : [a,b] \rightarrow \mathbb{R}$ is Riemann integrable, then $-f$ is and 
\begin{equation*}
\int_{a}^{b} (-f(x))\,dx = -\int_{a}^{b}f(x)\,dx
\end{equation*}

(iv) If $\lambda \in \mathbb{R}$ and $f : [a,b] \rightarrow \mathbb{R}$ is Riemann integrable, then $\lambda f$ is Riemann integrable and
\begin{equation*}
\int_{a}^{b} \lambda f(x)\,dx = \lambda \int_{a}^{b}f(x)\,dx
\end{equation*}

(v) If $f,g:[a,b]\rightarrow\mathbb{R}$ are Riemann integrable functions with $f(t)\geq g(t)$ for all $t\in[a,b]$, then 
\begin{equation*}
\int_{a}^{b}f(x)\,dx \geq \int_{a}^{b}g(x)\,dx
\end{equation*}
\end{AYLemma}

\begin{proof}
i) $S(1, \mathcal{D}) = \sum(x_j - x_{j-1})1 = x_n - x_{n-1} + x_{n-1} - x_{n-2} + \cdots - x_0 = x_n - x_0 = b-a$, and similarly $s(1, \mathcal{D}) = b-a$, so $I^*1 = b-a = I_*1$ so 1 is integrable $\int_a^b 1 = b-a$

ii) Suppose $f, g$ integrable on $[m, l]$. Then given $\epsilon > 0$ we can find a dissection $\mathcal{D}_1$ such that $S(f, \mathcal{D}_1) - s(f, \mathcal{D}_1) < \epsilon$ and a dissection $\mathcal{D}_2$ such that $S(g, \mathcal{D}_2) - s(g, \mathcal{D}_2) < \epsilon$

Take $\mathcal{D} = \mathcal{D}_1 \cup \mathcal{D}_2$
\begin{equation*}
S(f, \mathcal{D}_1) \geq S(f, \mathcal{D}) \geq If \geq s(f, \mathcal{D}) \geq s(f, \mathcal{D}_1)
\end{equation*}
so $S(f, \mathcal{D}) - s(f, \mathcal{D}) < \epsilon$, and similarly $S(g, \mathcal{D}) - s(g, \mathcal{D}) < \epsilon$

\begin{align*}
S(f+g, \mathcal{D}) &= \sum(x_j - x_{j-1}) \sup_{t \in [x_{j-1}, x_j]} (f(t) + g(t))\\
&\geq \sum(x_j - x_{j-1}) \sup_{t \in [x_{j-1}, x_j]} f(t) + \sup_{t \in [x_{j-1}, x_j]} g(t)\\
&= S(f, \mathcal{D}) + S(g, \mathcal{D})\\
&\geq s(f, \mathcal{D}) + s(g, \mathcal{D})\\
\geq s(f+g, \mathcal{D}) \text{ similarly}
\end{align*}

So
\begin{align*}
0 &\leq S(f+g, \mathcal{D}) + s(f+g, \mathcal{D})\\
&\leq (S(f, \mathcal{D}) - s(f, \mathcal{D})) + (S(g, \mathcal{D}) - s(g, \mathcal{D}))\\
&< 2 \epsilon
\end{align*}

so since $\epsilon > 0$ is arbitrary $f+g$ is integrable. Moreover
\begin{align*}
I(f+g) &\geq s(f, \mathcal{D}) + s(g, \mathcal{D})\\
&\geq (If - \epsilon) + (Ig - \epsilon)\\
&=If + Ig - 2\epsilon
\end{align*}

and sum to get $If + Ig + 2 \epsilon \geq I(f+g)$, so since this is true $\forall \epsilon > 0 \ If + Ig = I(f+g)$
\begin{equation*}
\int_a^bf + \int_a^bg = \int_a^b(f+g)
\end{equation*}

iii) Observe that
\begin{equation*}
S(-f, \mathcal{D}) = -s(f, \mathcal{D}) \qquad \text{so} \qquad I^*(-f) = -I_*(f)
\end{equation*}
\begin{equation*}
s(-f, \mathcal{D}) = -S(f, \mathcal{D}) \qquad \text{so} \qquad I_*(-f) = -I^*(f)
\end{equation*}
so $f$ integrable $\Rightarrow -f$ integrable

iv) Consider first the case when $\lambda \geq 0$
\begin{align*}
S(\lambda f,\mathcal{D}) &= \lambda S(f, \mathcal{D})\\
\text{so } I^*(\lambda f) &= \lambda I^*f\\
\text{similarly }I_*(\lambda f) &= \lambda I_*f
\end{align*}

so if $f$ is integrable so is $\lambda f$ and $I(\lambda f) = \lambda If$

For the case $\lambda < 0$ consider $|\lambda| (-f) = \lambda f$ and use the case $\lambda \geq 0$

v) If $f \geq g$ then $S(f, \mathcal{D}) \geq S(g, \mathcal{D})$ so $I^*f \geq I^*g$ so if $f$ and $g$ are integrable $If \geq Ig$
\end{proof}

\begin{AYLemma}
(i) If $f : [a,b] \rightarrow \mathbb{R}$ is Riemann integrable then so is $f^{2}$.

(ii) If $f, g : [a,b] \rightarrow \mathbb{R}$ are Riemann integrable, then so is the product $fg$. 
\end{AYLemma}

\begin{proof}
i) Suppose $|f(t)| \leq M \ \forall t$. Then
\begin{align*}
|f(t)^2 - f(s)^2| &= |f(t) - f(s)||f(t) + f(s)|\\
&\leq |f(t) - f(s)|(|f(t)| + |f(s)|)\\
&= 2M |f(t) - f(s)|
\end{align*}
so $\sup_{t \in J} f(t)^2 - \inf_{s \in J} f(s)^2 \leq 2M (\sup_{t \in J} f(t) - \inf_{s \in J} f(s))$. Thus
\begin{align*}
S(f^2, \mathcal{D}) - s(f^2, \mathcal{D}) &= \sum(x_j - x_{j-1}) \sup_{t \in [x_{j-1}, x_j]} f(t)^2 - \inf_{s \in [x_{j-1}, x_j]} f(s)^2\\
&\leq 2M \sum (x_j - x_{j-1} (\sup_{t \in [x_{j-1}, x_j]} f(t) - \inf_{s \in [x_{j-1}, x_j]} f(s))\\
&= 2M(S(f, \mathcal{D}) - s(f, \mathcal{D}))
\end{align*}
If $f$ is integrable then given any $\epsilon > 0$ we can find $\mathcal{D}$ such that $S(f, \mathcal{D}) - s(f, \mathcal{D}) < \frac{\epsilon}{2M+1}$ and so $S(f^2, \mathcal{D}) - s(f^2, \mathcal{D}) < \epsilon$ so $f^2$ is integrable

iii) If $f, g$ are integrable then $f+g, f-g$ are integrable so $(f+g)^2$ and $(f-g)^2$ are integrable, so $4fg = (f+g)^2 - (f-g)^2$ is integrable, and thus $fg = \frac{1}{4} (4fg)$ is integrable
\end{proof}

\begin{AYLemma}
(i) If $f : [a,b] \rightarrow \mathbb{R}$ is Riemann integrable then so is $f_{+}(t) = \max(f(t),0)$.

(ii) If $f : [a,b] \rightarrow \mathbb{R}$ is Riemann integrable, then $|f|$ is Riemann integrable and
\begin{equation*}
\int_{a}^{b} |f(x)|\,dx \geq \left|\int_{a}^{b}f(x)\,dx\right|
\end{equation*}
\end{AYLemma}

\begin{proof}
i) $|f(t) - f(s)| \geq |f_+(t) - f_+(s)|$ so $\sup_{t \in J}(f(t)) - \inf_{s \in J} (f(s)) \geq \sup_{t \in J} f_+(t) - \inf_{s \in J} f_+(s)$

so along the same lines as the previous lemma $S(f, \mathcal{D}) - s(f, \mathcal{D}) \geq S(F_+, \mathcal{D}) - s(f_+, \mathcal{D})$, so if $f$ is integrable, given $\epsilon > 0, \exists \mathcal{D}$ such that $S(f, \mathcal{D}) - s(f, \mathcal{D}) < \epsilon$ so $S(f_+, \mathcal{D}) - s(f_+, \mathcal{D}) < \epsilon$. Thus $f_+$ is integrable

ii) Write

\begin{align*}
f_+(t) &= f(t) \quad \text{for }f(t) \geq 0\\
&= 0 \quad \text{for }f(t) < 0\\
f_-(t) &= 0 \quad \text{for }f(t) \geq 0\\
&= -f(t) \quad \text{for }f(t) < 0\\
\end{align*}

Then $f = f_+ - f_-$ and $|f| = f_+ + f_-$

By i) if $f$ is integrable so are $f_+$ and $f_-$, so $|f| = f_+ + f_-$ is integrable

$|f| = f_+ + f_-$ so $\int_a^b|f| = \int f_+ + \int f_-$, $\int_a^b f = \int f_+ - \int f_-$

$\int f_+, \int f_- \geq 0$ so $\int_a^b |f| \geq |\int_a^b f|$
\end{proof}

Notice that we often only need the much weaker inequality
\begin{equation*}
\left|\int_{a}^{b}f(x)\,dx\right|\leq \sup_{t\in[a,b]}|f(t)|(b-a)\] usually stated as \[|\text{integral}|\leq\text{length}\times\text{sup}
\end{equation*}

(For the simplest proof, write $M = \sup|f|$, then $M \geq f(s) \geq -M \ \forall s \in [a,b]$

so $M(b-a) \geq S(f, \mathcal{D}) \geq s(f, \mathcal{D}) \geq -M(b-a)$

so $M(b-a) \geq I^*f \geq I_*f \geq -M(b-a)$

so if $If$ exists, $|If| \geq M(b-a)$

The next lemma is also routine in its proof but continues our programme of showing that the integral has all the properties we expect.

\begin{AYLemma}
\label{RiemannOverTwoRanges}
Suppose that $a \leq c \leq b$ and that $f : [a, b] \rightarrow \mathbb{R}$ is a bounded function.  Then, $f$ is Riemann integrable on $[a, b]$ if and only if $f|_{[a, c]}$ is Riemann integrable on $[a, c]$ and $f|_{[c, b]}$ is Riemann integrable on $[c, b]$. Further, if $f$ is Riemann integrable on $[a, b]$, then
\begin{equation*}
\int_{a}^{b}f(x)\,dx=\int_{a}^{c}f|_{[a,c]}(x)\,dx +\int_{c}^{b}f|_{[c,b]}(x)\,dx
\end{equation*}
\end{AYLemma}

\begin{proof}
Suppose $f: [a, b] \rightarrow \mathbb{R}$ and $a < c < b$. If $\mathcal{D}_1$ is a dissection of $[a, c]$ with $a = x_0 \leq x_1 \leq \ldots \leq x_n = c$ and $\mathcal{D}_2$ is a dissection of $[c, b]$ with $c = x_{n+1} \leq x_{n+1} \leq \ldots \leq x_m = b$
\begin{align*}
S_{[a, b]}(f, \mathcal{D}_1 \cup \mathcal{D}_2) &= \sum(x_j - x_{j-1}) \sup_{t \in [a, b]} f(t)\\
&= S_{[a, c]} (f, \mathcal{D}_1) + S_{[c, b]} (f, \mathcal{D}_2)
\end{align*}
and so $I^*_{[a, b]} f \leq I^*_{[a, c]} f + I^*_{[c, b]} f$

On the other hand if $\mathcal{D}$ is a dissection of $[a, b]$
\begin{align*}
S_{[a, b]}(f, \mathcal{D}) &\geq S_{[a, b]} (f, \mathcal{D} \cup \{c\})\\
&= S_{[a, c]} (f, \mathcal{D}_1) + S_{[c, b]} (f, \mathcal{D}_2)
\end{align*}

where $\mathcal{D}_1$ is a dissection of $[a, c]$ and $\mathcal{D}_2$ is a dissection of $[c, b]$, and thus $\mathcal{D}_1 \cup \mathcal{D}_2 = \mathcal{D} \cup \{c\}$

so $I_{[a, b]}^*f \geq I_{[a, c]}^*f + I_{[c, b]}^*f$

$\therefore I_{[a, b]}^*f = I_{[a, c]}^*f + I_{[c, b]}^*f$

Similarly ${I_*}_{[a, b]}f = {I_*}_{[a, c]}f + {I_*}_{[c, b]}f$

So if $f$ is integrable over $[a, c], [c, b]$ which gives us
\begin{equation*}
I^*_{[a, c]} f = {I_*}_{[a,c]} f = I_{[a, c]}f \qquad I^*_{[c, b]} f = {I_*}_{[c,b]} f = I_{[c, b]}f
\end{equation*}
and thus
\begin{equation*}
I^*_{[a, b]} f = {I_*}_{[a,b]} f = I_{[a, b]}f
\end{equation*}

Conversely, if $f$ is integrable over $[a, b]$

\begin{align*}
I_{[a, b]}f &= I^*_{[a,b]} f = I^*_{[a, c]}f + I^*_{[c, b]}f\\
&\geq {I_*}_{[a, c]} f + {I_*}_{[c, b]} f = {I_*}_{[a, b]} f\\
&= I_{[a, b]} f
\end{align*}
so $f$ is integrable on $[a, c], [c, b]$ with
\begin{equation*}
I_{[a, b]}f = I_{[a, c]}f + I_{[c, b]}f
\end{equation*}
\end{proof}

In a very slightly less precise and very much more usual notation we write
\begin{equation*}
\int_{a}^{b}f(x)\,dx = \int_{a}^{c}f(x)\,dx + \int_{c}^{b}f(x)\,dx
\end{equation*}

There is a standard convention which says that, if $b\geq a$ and $f$ is Riemann integrable on $[a,b]$, we define \begin{equation*}
\int_{b}^{a}f(x)\,dx=-\int_{a}^{b}f(x)\,dx
\end{equation*}
It is, however, a convention that requires care in use. 

\begin{AYExercise}
Suppose that $b \geq a$, $\lambda, \mu \in \mathbb{R}$, and $f$ and $g$ are Riemann integrable. Which of the following statements are always true and which are not? Give a proof or counter-example. If the statement is not always true, find an appropriate correction and prove it.

(i) ${\displaystyle \int_{b}^{a} \lambda f(x) + \mu g(x)\,dx =\lambda \int_{b}^{a}f(x)\,dx + \mu \int_{b}^{a}g(x)\,dx}$.

(ii) If $f(x) \geq g(x)$ for all $x \in[a, b]$, then ${\displaystyle \int_{b}^{a}f(x)\,dx \geq \int_{b}^{a}g(x)\,dx}$. 
\end{AYExercise}

\section{Some properties of the integral}

Not all bounded functions are Riemann integrable 

\begin{AYLemma}
If $f : [0, 1] \rightarrow \mathbb{R}$ is given by
\begin{align*}
f(x) = 1 &\qquad \text{when $x$ is rational,}\\
f(x) = 0 &\qquad \text{when $x$ is irrational,}
\end{align*}
then $f$ is not Riemann integrable
\end{AYLemma}

This does not worry us unduly, but makes it more important to show that the functions we wish to be integrable actually are.

Our first result goes back to Riemann (indeed, essentially, to Newton and Leibniz). 

\begin{AYLemma}
\label{RiemannIncreasingCriterion}
(i) If $f : [a, b] \rightarrow \mathbb{R}$ is increasing, then $f$ is Riemann integrable.
 
(ii)If $f : [a, b] \rightarrow \mathbb{R}$ can be written as $f = f_1 - f_2$ with $f_1, f_2 : [a,b] \rightarrow \mathbb{R}$
increasing, then $f$ is Riemann integrable.

(iii) If $f : [a, b] \rightarrow \mathbb{R}$ is piecewise monotonic, then $f$ is Riemann integrable.
\end{AYLemma}

\begin{proof}
i) We show that given $\epsilon \ \exists$ a $\mathcal{D}$ such that $S(f, \mathcal{D}) -s(f, \mathcal{D}) < \epsilon$ (the hard bit is to find the dissection)

Choose $\mathcal{D}$ to be $a, a + \frac{b-a}{N}, a + \frac{2(b-a)}{N}, \ldots, b$

\begin{align*}
S(f, \mathcal{D}) - s(f, \mathcal{D}) &= \sum (\sup_{t \in [a + \frac{(b-a)(r-1)}{N}, a + \frac{(b-a)r}{N}]} f(t) - \inf_{t \in [a + \frac{(b-a)(r-1)}{N}, a + \frac{(b-a)r}{N}]} f(t))\\
&= \sum(f(a + \frac{(b-a)r}{N}) - f(a + \frac{(b-a)(r-1)}{N})).\frac{b-a}{N}\\
&=(f(b)-f(a))\frac{b-a}{N}
\end{align*}

By choosing $N$ large enough we have $(f(b) - f(a)) \frac{b-a}{N} < \epsilon$

ii) This means $f$ is the sum of an increasing and decreading function (both Riemann integrable), so thus it is Riemann integrable

iii) Using Lemma $\ref{RiemannOverTwoRanges}$ and induction
\end{proof}

It should be noted that the results of  Lemma~\ref{RiemannIncreasingCriterion} do not require $f$ to be continuous. (For example, the Heaviside function, given by $H(t)=0$ for $t<0$, $H(t)=1$ for $t\geq 0$ is increasing but not continuous.) It is quite hard to find a continuous function which is not the difference of two increasing functions but  an example is given in
question 18 on the 4th example sheet.

The proof of the next result is starred. Next year you will see a simpler proof based on a different idea (that of uniform continuity).

\begin{AYTheorem}
If $f : [a, b] \rightarrow \mathbb{R}$ is continuous then $f$ is Riemann integrable.
\end{AYTheorem}

\begin{proof}
Suppose that $f : [a, b] \rightarrow \mathbb{R}$ is not integrable. Then there exists a $\delta > 0$ such that $I^*_{[a, b]}f - {I_*}_{[a, b]}f = \delta(b-a)$

Set $a_0 = a, b_0 = b$, $c_0 = c = \frac{a_0 + b_0}{2}$

\begin{equation*}
I^*_{[a, c]}f - I^*_{[c, b]}f = I^*_{[a, b]}f \qquad {I_*}_{[a, c]}f - {I_*}_{[c, b]}f = {I_*}_{[a, b]}f
\end{equation*}
so
\begin{align*}
\max\{I^*_{[a, c]}f - {I_*}_{[a, c]}f, I^*_{[c, b]}f - {I_*}_{[c, b]}f\} &\geq \frac{1}{2} (I^*_{[a, b]}f - {I_*}_{[a, b]}f)\\
&=\frac{1}{2} \delta(b-a)
\end{align*}

Choose $a_1 = a_0, b_1 = c_0$ if $I^*_{[a, c]}f - {I_*}_{[a, c]}f \geq \frac{1}{2} \delta(b-a)$, otherwise $a_1 = c_0, b_1 = b_0$

Observe
\begin{quote}
$a_0 \leq a_1 \leq b_1 \leq b$

$I^*_{[a_1, b_1]}f - {I_*}_{[a_1, b_1]}f \geq (b_1 - a_1) \delta$

$b_1 - a_1 = 2^{-1} (b_0 - a_0)$
\end{quote}

Extend this by induction to get

\begin{quote}
$a_0 \leq a_1 \leq \ldots \leq a_n \leq b_n \ldots \leq b_1 \leq b_0$

$I^*_{[a_n, b_n]}f - {I_*}_{[a_n, b_n]}f \geq (b_n - a_n) \delta$

$b_n - a_n = 2^{-n} (b_0 - a_0)$
\end{quote}

By the same arguments that we have made before, the $a_n$ are increasing, bounded above, and so $a_n$ tend to a limit $\alpha$, and as $b_n - a_n \rightarrow 0$ so $b_n \rightarrow \alpha$, and further $b_n \leq b$ so $\alpha \leq b$ and $a_n \leq \alpha$ so $a \leq \alpha$

Examine what happens at $\alpha$

$f$ is continuous at $\alpha$ so $\exists \eta > 0$ such that $|f(x) - f(\alpha)| \leq \frac{\delta}{4} \ \forall x \in [a, b], |x-\alpha| \leq \eta$

Pick $n$ such that $|a_n \alpha|, |b_n \alpha| < \eta$

Then $f(\alpha) -\frac{\delta}{4} \leq f(x) \leq f(\alpha) + \frac{\delta}{4}$ for $x \in [a_n, b_n]$, so

\begin{align*}
I^*_{[a_n, b_n]}f - {I_*}_{[a_n, b_n]}f &\leq 2 \frac{\delta}{4} (b_n - a_n)\\
&= \frac{\delta}{2} (b_n - a_n)\\
&< \delta (b_n - a_n) \#
\end{align*}

So $I^*f = I_*f$ and $f$ is integrable
\end{proof}

We complete the discussion of integration and this course with a series of results which apply only to \emph{continuous} functions.

Our first result is an isolated, but useful, one. 

\begin{AYLemma}
If $f : [a, b] \rightarrow \mathbb{R}$ is continuous, $f(t)\geq 0$ for all $t\in[a,b]$ and 
\begin{equation*}
\int_{a}^{b}f(t)\,dt=0
\end{equation*}
it follows that $f(t) = 0$ for all $t \in [a,b]$.
\end{AYLemma}

\begin{proof}
Suppose $f:[a, b] \rightarrow \mathbb{R}$ is continuous and $f(t) \geq 0 \ \forall t \in [a, b]$

Suppose $\exists c \in [a, b]$ and $f(c) \neq 0$

Then $f(c) > 0$ and by continuity $\exists \delta > 0$ such that $|f(x) - f(c)| < \frac{f(c)}{2}$ for $|x-c| \leq \delta$ and $x \in [a, b]$, so $f(x) > \frac{f(c)}{2}$ is $|x-c| \leq \delta$ and $x \in [a, b]$

Write $J = \{a \in [a, b] : |x-c| \leq \delta\}$

$J$ is an interval, with the length of $J \geq \delta$

(note: we must allow for the possibility that $c=a$ or $c=b$)

\begin{figure}[htbp]
	\centering
		\includegraphics{Images/AY03.jpeg}
	\caption{Interval around a point}
	\label{Interval around a point}
\end{figure}

\begin{align*}
\int_a^b f(x)\, dx &= \int_J f(x) \, dx + \int_{[a, b]\setminus J} f(x) \, dx\\
&\geq \int_J f(x)\, dx\\
&\geq \text{length }J . \frac{f(x)}{2}\\
&\> 0
\end{align*}
\end{proof}

\begin{AYExercise}
Let $a \leq c \leq b$. Give an example of a Riemann integrable function $f : [a, b] \rightarrow \mathbb{R}$ such that $f(t) \geq 0$ for all $t \in [a, b]$ and
\begin{equation*}
\int_{a}^{b}f(t)\,dt=0
\end{equation*}
but $f(c) \neq 0$.
\end{AYExercise}

\begin{proof}
$f(c) = 1, f(t) = 0$ otherwise will do
\end{proof}

We now come to the justly named fundamental theorem of the calculus. 
\begin{AYTheorem}[The fundamental theorem of the calculus]
\label{FundamentalTheoremCalculus}
Suppose that $f : (a, b) \rightarrow \mathbb{R}$ is a continuous function and that $u \in (a, b)$. If we set 
\begin{equation*}
F(t) = \int_{u}^{t}f(x)\,dx
\end{equation*}
then $F$ is differentiable on $(a, b)$ and $F'(t) = f(t)$ for all $t \in (a, b)$.
\end{AYTheorem}

\begin{proof}
\begin{align*}
|\frac{F(t+h) - F(t)}{h} - f(t)| &= |\frac{1}{h} (\int_u^{t+h} f(x) \, dx - \int_u^t f(x) \, dx) - f(t)|\\
&=|\frac{1}{h} \int_t^{t+h} f(x) \, dx - f(t)|\\
&= |\frac{1}{h} \int_t^{t+h} (f(u) - f(t)) \, du| \quad (\int_t^{t+h} f(t) \, du = hf(t) \because f(t) \text{ constant wrt }u)\\
&\leq \frac{1}{|h|} \text{ length }.\sup\\
&= \frac{1}{|h|} |h| \sup_{|u-t| \leq |h|} |f(u) - f(t)|\\
&= \sup_{|u-t| \leq |h|} |f(u) - f(t)| \rightarrow 0 \text{ as } h \rightarrow 0 \because f \text{ is continuous}\\
\therefore F'(t) &= f(t)
\end{align*}
\end{proof}

\begin{AYExercise}
(i) Let $H$ be the Heaviside function $H : \mathbb{R} \rightarrow \mathbb{R}$ given by $H(x) = 0$ for $x < 0$, $H(x) = 1$ for $x \geq 0$. Calculate $F(t) = \int_{0}^{t}H(x)\,dx$ and show that $F$ is not differentiable at $0$. Where does our proof of Theorem~\ref{FundamentalTheoremCalculus} break down?

(ii) Let $f(0) = 1$, $f(t) = 0$ otherwise. Calculate $F(t) = \int_{0}^{t}f(x)\,dx$ and show that $F$ is differentiable at $0$ but $F'(0) \neq f(0)$. Where does our proof of Theorem~\ref{FundamentalTheoremCalculus} break down?
\end{AYExercise}

Sometimes we think of the fundamental theorem in a slightly different way. 

\begin{AYTheorem}
Suppose that $f : (a, b) \rightarrow \mathbb{R}$ is continuous, that $u \in(a,b)$ and $c \in \mathbb{R}$. Then there is a unique solution to the differential equation $g'(t) = f(t)$ $[t \in (a, b)]$ such that $g(u) = c$.
\end{AYTheorem}

\begin{proof}
First, prove existence

Let $g(t) = \int_u^t f(x) \, dx$

By the fundamental theorem of the calculus $g'(t) = f(t)$

Observe $g(u) = 0 + c = c$

Next, show uniqueness

If $h'(t) = g'(t) = f(t)$ with $h(u) = g(u) = c$ set $a(t) = h(t) - g(t)$, and thus $a'(t) = h'(t) - g'(t) = f(t) - f(t) = 0 \ \forall t$, so by the Mean Value Theorem $a(t)$ is constant

$\therefore a(u) = h(u) - g(u) = c -c =0$ so $a(t) = 0 \ \forall t$ so $h(t) = g(t)$
\end{proof}

We call the solutions of $g'(t) = f(t)$ \emph{indefinite integrals} (or, simply, \emph{integrals}) of $f$.

Yet another version of the fundamental theorem is given by the next theorem.

\begin{AYTheorem}
\label{IntegrationDifferentiationTheorem}
Suppose that $g : [a, b] \rightarrow \mathbb{R}$ has continuous derivative. Then
\begin{equation*}
\int_{a}^{b}g'(t)\,dt = g(b)-g(a)
\end{equation*}
\end{AYTheorem}

\begin{proof}
Set $G(s) = \int_a^s g'(t) /. dt - g(s)$

$G'(s) = g'(s) - g'(s)$ so by the Mean Value Theorem $G$ is constant, so $G(b) = G(a)$ i.e. $\int_a^b g'(t) \, dt - g(b) = -g(a)$ so $\int_a^b g'(t) \, dt = g(b) - g(a)$
\end{proof}

Theorems~\ref{FundamentalTheoremCalculus} and~\ref{IntegrationDifferentiationTheorem} show that (under appropriate circumstances) integration and differentiation are inverse operations and the the theories of differentiation and integration are subsumed in the greater theory of the calculus.
 
We use the fundamental theorem of the calculus to prove the formulae for integration by substitution and integration by parts.

\begin{AYTheorem}[Change of variables for integrals]
Suppose that $f : [a, b] \rightarrow \mathbb{R}$ is continuous and $g : [\gamma,\delta] \rightarrow \mathbb{R}$ is differentiable with continuous derivative. Suppose further that $g([\gamma,\delta]) \subseteq [a,b]$. Then, if $c, d \in [\gamma,\delta]$, we have 
\begin{equation*}
\int_{g(c)}^{g(d)}f(s)\,ds = \int_{c}^{d}f(g(x))g'(x)\,dx
\end{equation*}
\end{AYTheorem}

\begin{proof}
Rearrange to $\int_c^d f(g(x)) g'(x) \, dx - \int_{g(c)}^{g(t)} f(s) \, ds = 0$, and replace by the more general problem

\begin{equation*}
\int_c^t f(g(x)) g'(x) \, dx - \int_{g(c)}^{g(t)} f(s) \, ds = 0 \quad \forall c \leq t \leq d
\end{equation*}
Proof begins:
\begin{equation*}
\frac{d}{dt} (\int_c^t f(g(x)) g'(x) \, dx - \int_{g(c)}^{g(t)} f(s) \, ds) = f(g(t)) g'(t) - g'(t) f(g(t)) = 0
\end{equation*}
(by the chain rule, $\frac{d}{dt} f(g(t)) = g'(t) f'(g(t))$)

so $\int_c^t f(g(x)) g'(x) \, dx - \int_{g(c)}^{g(t)} f(s) \, ds$ is constant (MVT)

Taking $t = c$ we see that the constant is zero, so

\begin{equation*}
\int_c^t f(g(x)) g'(x) \, dx - \int_{g(c)}^{g(t)} f(s) \, ds = 0
\end{equation*}
\end{proof}

\begin{AYExercise}
The following exercise is traditional.

(i) Show that integration by substitution, using $x = 1/t$, gives
\begin{equation*}
\int_{a}^{b}\frac{dx}{1 + x^{2}} = \int_{1/b}^{1/a}\frac{dt}{1 + t^{2}}
\end{equation*}
when $b > a > 0$.

(ii) If we set $a = -1$, $b = 1$ in the formula of~(i), we obtain
\begin{equation*}
\int_{-1}^{1}\frac{dx}{1 + x^{2}} \overset{?}{=} -\int_{-1}^{1}\frac{dt}{1 + t^{2}}
\end{equation*}
Explain this apparent failure of the method of integration by substitution.

(iii) Write the result of (i) in terms of $\tan^{-1}$ and prove it using standard trigonometric identities.
\end{AYExercise}

\begin{AYLemma}
Suppose that $f : [a, b] \rightarrow \mathbb{R}$ has continuous derivative and $g : [a, b] \rightarrow \mathbb{R}$ is continuous. Let $G : [a, b] \rightarrow \mathbb{R}$ be an indefinite integral of $g$. Then, we have 
\begin{equation*}
\int_{a}^{b}f(x)g(x)\,dx=\left[f(x)G(x)\right]_{a}^{b} -\int_{a}^{b}f'(x)G(x)\,dx
\end{equation*}
\end{AYLemma}

\begin{proof}
\begin{align*}
&\frac{d}{dt} (\int_a^t f(x) g(x) \, dx - (f(t)G(t) - f(a)G(a)) + \int_a^t f'(x) G(x)) \ dx \\
& \quad = f(t)g(t) - (f'(t)G(t) + f(t) G'(t)) + f'(t) G(t) = 0
\end{align*}

So by the MVT $\int_a^t f(x) g(x) \, dx - (f(t)G(t) - f(a)G(a)) + \int_a^t f'(x) G(x)$ is constant. Setting $t=a$ we see that the constant is zero, and the result follows
\end{proof}

We obtain the following version of Taylor's theorem by repeated integration by parts. 

\begin{AYTheorem}[A global Taylor's theorem with integral remainder]
If $n \geq 1$ and $f : (-a, a) \rightarrow \mathbb{R}$ is $n$ times differentiable with continuous $n$th derivative, then 
\begin{equation*}
f(t) = \sum_{j = 0}^{n - 1}\frac{f^{(j)}(0)}{j!}t^{j} + R_n(f,t)
\end{equation*}
for all $|t| < a$, where
\begin{equation*}
R_n(f,t) = \frac{1}{(n-1)!} \int_{0}^{t}(t-x)^{n-1}f^{(n)}(x)\,dx
\end{equation*} 
\end{AYTheorem}

\begin{proof}
$R_n = \frac{1}{(n-1)!} \int_0^t (t-x)^{n-1} f^{(n)} (x) \, dx$

Integrate by parts:

\begin{align*}
R_n &= \left[\frac{(t-x)^{n-1}}{(n-1)!} f^{(n-1)}(x)\right]_0^t + \frac{1}{(n-2)!} \int_0^t (t-x)^{n-1} f^{(n-1)}(x) \, dx\\
&= \frac{-f^{(n-1)}(0) t^{n-1}}{(n-1)!} + R_{n-1}\\
&= -\frac{f^{(n-1)}(0)t^{n-1}}{(n-1)!} - \frac{f^{(n-2)}(0)t^{n-2}}{(n-2)!} + R_{n-2}\\
&= -\frac{f^{(n-1)}(0)t^{n-1}}{(n-1)!} - \cdots - {f'(0)t}{1!} + \underbrace{\int_0^t f'(x) \, dx}_{R_1}\\
&= -\frac{f^{(n-1)}(0)t^{n-1}}{(n-1)!} - \cdots - {f'(0)t}{1!} + f(t) - f(0)
\end{align*}
Rearranging:
\begin{equation*}
f(t) = f(0) + \frac{f'(0)t}{1!} + \cdots + \frac{f^{(n-1)}(0) t^{n-1}}{(n-1)!} + R_n(f, t)
\end{equation*}
\end{proof}

In the opinion of the lecturer this form is powerful enough for most purposes and is a form that is easily remembered and proved for examination.

\section{Infinite integrals}

The reader may already be familiar with definitions of the following type.

\begin{AYDefinition}
\label{InfiniteIntegrationDefinition} 
If $f:[a,b]\rightarrow\mathbb{R}$ and $M,P\geq 0$ let us write 
\begin{equation*}
f_{M,P}(x)= 
\begin{cases}
f(x) & \text{if $-P \leq f(x) \leq M$}\\
M & \text{if $f(x) > M$}\\
-P & \text{if $f(x) < -P$.}
\end{cases}
\end{equation*}
If $f_{M,P}$ is Riemann integrable for each $M, P \geq 0$ and
\begin{equation*}
\int_{a}^{b}f_{M,P}(x)\,dx\rightarrow L
\end{equation*}
as $M, P \rightarrow \infty$ then we say that $f$ is Riemann integrable and 
\begin{equation*}
\int_{a}^{b}f(x)\,dx=L
\end{equation*}
\end{AYDefinition}

\begin{AYDefinition}
\label{UpperLimitIntegrationDefinition}
If $f:[a, \infty) \rightarrow \mathbb{R}$ is such that $f|_{[a, X]}$ is Riemann integrable for each $X > a$ and $\int_{a}^{X}f(x)\,dx \rightarrow L$ as $X \rightarrow \infty$, then we say that $\int_{a}^\infty f(x)\,dx$ exists with value $L$.
\end{AYDefinition}

It must be said that neither Definition~\ref{InfiniteIntegrationDefinition} nor Definition~\ref{UpperLimitIntegrationDefinition} are more than ad hoc. 

In the rest of this section we look at Definition~\ref{UpperLimitIntegrationDefinition}.
\begin{AYLemma}
\label{BoundedFunctionIntegrable} 
Suppose $f : [a, \infty) \rightarrow \mathbb{R}$ is such that $f|_{[a,X]}$ is Riemann integrable on $[a, X]$ for each $X > a$. If $f(x) \geq 0$ for all $x$, then $\int_{a}^\infty f(x)\,dx$ exists if and only if there exists a $K$ such that $\int_{a}^{X}f(x)\,dx \leq K$ for all $X$.
\end{AYLemma}

\begin{proof}
Set $u_n = \int_a^n f(x) \, dx$, then $u_{n+1} - u_n = \int_n^{n+1} f(x) \, dx \geq 0$ (as $f \geq 0$)

so $u_n$ is an increasing sequence bounded above by the hypothesis, so $u_n$ tends to a limit, $L$ say.

Thus given $\epsilon > 0 \ \exists N(\epsilon)$ such that $L \geq u_n \geq L - \epsilon \ \forall n \geq N(\epsilon)$ If $X \geq N(\epsilon)$ we can choose $m$ so that $m+1 \geq X \geq m$, with $u_{m+1} \geq u_X \geq u_m$ so $L \geq u_X \geq L - \epsilon$ and $u_X \rightarrow L$ as $X \rightarrow \infty$
\end{proof}

We use Lemma~\ref{BoundedFunctionIntegrable} to prove the integral comparison test.

\begin{AYLemma}
\label{IntegralTestLemma}
If $f : [1, \infty) \rightarrow \mathbb{R}$ is a decreasing function with $f(x) \rightarrow 0$ as $x \rightarrow \infty$ then 
\begin{equation*}
\sum_{n=1}^\infty f(n) \text{ and } \int_1^\infty f(x)\,dx
\end{equation*}
either both diverge or both converge.
\end{AYLemma}

\begin{figure}[htbp]
	\centering
		\includegraphics{Images/AY04.jpeg}
	\caption{Integration of a decreasing function}
	\label{Integration of a decreasing function}
\end{figure}

\begin{proof}
$f(n-1) \geq f(x) \geq f(n)$ for $n \geq x \geq (n-1)$

\begin{equation*}
\int_{n-1}^n f(n-1) \, dx \geq \int_{n-1}^n f(x) \, dx \geq \int_{n-1}^n f(n) \, dx
\end{equation*}

i.e. $f(n-1) \geq \int_{n-1}^n f(x) \, dx \geq f(n)$. Sum from $n=2$ to $N+1$

\begin{equation*}
\sum_{n=2}^{N+1} f(n-1) \geq \sum_{n=2}^{N+1} \int_{n-1}^n f(x) \, dx \geq \sum_{n=2}^{N+1} f(n)
\end{equation*}
\begin{equation*}
\therefore \sum_1^N f(n) \geq \int_1^{N+1} f(x) \, dx \geq \sum_2^{N+1} f(n)
\end{equation*}

If $\sum f(n)$ converges then $\sum_1^N f(n)$ is bounded
\begin{equation*}
\sum_1^N f(n) \leq A \ \forall N
\end{equation*}

So $A \geq \int_1^{N+1} f(x) \, dx \ \forall N \geq 1$

so since $f$ is positive, $A \geq \int_1^X f(x) \, dx \ \forall X \in \mathbb{R}, X \geq 1$, so $\int_1^\infty f(x) \, dx$ converges.

Conversely, if $\int_1^\infty f(x) \, dx$ converges $\exists B$ such that $B \geq \int_1^X f(x) \, dx \ \forall X$ so $B \geq \sum_2^{N+1} f(n) \ \forall N$ such that $B + f(1) \geq \sum_1^M f(r) \ \forall M$ so $\sum f(n)$ converges
\end{proof}

\begin{AYExample}
If $\alpha > 1$ then $\sum_{n=1}^\infty n^{-\alpha}$ converges. If $\alpha\leq 1$ then $\sum_{n=1}^\infty n^{-\alpha}$ diverges.
\end{AYExample}

\begin{proof}
\begin{align*}
\int_1^X x^{-\alpha} \, dx &= \left[\frac{x^{1-\alpha}}{1-\alpha}\right]^X_1 \quad \alpha \neq 1\\
&= \frac{X^{1-\alpha}-1}{1-\alpha}
\end{align*}
Which tends to $\infty$ if $\alpha < 1$ and converges to $\frac{1}{1-\alpha}$ if $\alpha > 1$

Thus $\sum n^{-\alpha}$ diverges for $\alpha < 1$ and converges for $\alpha > 1$

($x^{-1\alpha}$ is decreasing for $\alpha > 0$, $\alpha \leq 0$ is trivial, $x^{-\alpha} \cancel{\rightarrow} 0$)

In the case $\alpha = 1$, $\frac{1}{x}$ is a decreasing positive function, and $\int_1^X \frac{1}{x} \, dx = \log X \rightarrow \infty$ so $\sum \frac{1}{n}$ diverges
\end{proof}

Along the same lines
\begin{equation*}
\int_\xi^X \frac{1}{x (\log x)^\alpha} \, dx = \left[\frac{(\log x)^{1- \alpha}}{1-\alpha}\right]_\xi^X
\end{equation*}
which converges if $\alpha >1$ and diverges if $\alpha <1$

\begin{equation*}
\int_\xi^X \frac{1}{x \log x} \, dx = \left[\log (\log x)\right]_\xi^X \rightarrow \infty \text{ as }X \rightarrow \infty
\end{equation*}
$\sum_{n=\xi}^\infty \frac{1}{X(\log X)^\beta}$ converges if $\beta > 1$ and diverges if $\beta \leq 1$

This is really as far as we need to go, but I will just add one further remark. 

\begin{AYLemma}
\label{AbsoluteConvergenceImpliesConvergence}
Suppose $f : [a, \infty) \rightarrow \mathbb{R}$ is such that $f|_{[a, X]}$ is Riemann integrable on $[a, X]$ for each $X > a$. If $\int_{a}^\infty|f(x)|\,dx$ exists, then $\int_{a}^\infty f(x)\,dx$ exists.
\end{AYLemma}

\begin{proof}
Let
\begin{align*}
f_+(x) &= f(x) \text{ when }f(x) \geq 0\\
&= 0 \text{ when }f(x) \leq 0
f_-(x) &= -f(x) \text{ when }f(x) \leq 0\\
&= 0 \text{ when }f(x) \geq 0
\end{align*}

$f_+, f_-$ are locally integrable with $|f| \geq f_+ \geq 0$ so $\int_a^x |f| \geq \int_a^x f_+ \geq 0$

But $\int_a^\infty |f|$ converges such that $\int_a^X |f| \leq A \ \forall X$ for some $A$, so $\int_a^X f_+ \leq A \ \forall X$ i.e. $\int_a^\infty f_+(x)\, dx$ tends to a limit $\int_a^\infty f_+(x) \, dx$

$\int_a^\infty f_-$ exists for the same reason, and 
\begin{equation*}
\int_a^X f = \int_a^X f_+ - f_- \rightarrow \int_a^\infty f_+ - \int_a^\infty f_-
\end{equation*}
and thus $\int_a^\infty f$ exists
\end{proof}

It is natural to state Lemma~\ref{AbsoluteConvergenceImpliesConvergence} in the form `absolute convergence of the integral implies convergence'. 

Speaking broadly, infinite integrals $\int_{a}^\infty f(x)\,dx$ work well when they are absolutely convergent, that is to say,
$\int_{a}^\infty|f(x)|\,dx<\infty$, but are full of traps for the unwary otherwise. This is not a weakness of the Riemann integral but inherent in any mathematical situation where an object only exists `by virtue of the cancellation of two infinite objects'. (Question 17 on the 4th example sheet gives an example of an integral which is convergent but not absolutely convergent.)

(The final lemma and example were given in lectures, but did not form part of the printed notes)

\begin{AYLemma}
If $f, g$ are locally Riemann-integrable and $f \geq g \geq 0$
\begin{equation*}
\int_a^\infty f \text{ converges }\rightarrow \int_a^\infty g \text{ converges}
\end{equation*}
\end{AYLemma}

\begin{proof}
Similar to the proof of the preceding lemma. As $f \geq g \geq 0, \int_a^X f(x) \geq \int_a^X g(x) \geq 0$

Thus $\int_a^\infty f$ converges $\Rightarrow \int_a^X f$ is bounded $\Rightarrow \int_a^X g$ is bounded and thus $\int_a^\infty g$ exists
\end{proof}

\begin{AYExample}
\begin{equation*}
\int_{-\infty}^{+\infty} \exp \left(-\frac{x^2}{2}\right) \text{ exists}
\end{equation*}
\end{AYExample}

\begin{proof}
$\exp \left(-\frac{x^2}{2}\right) \leq \exp \left(-\frac{x}{2}\right)$ for $x \geq 1$
\begin{equation*}
\int_1^X \exp \left(-\frac{x}{2}\right) = \frac{1}{2} (e^{-\frac{1}{2}} - e^{-\frac{x}{2}}) \rightarrow \frac{1}{2} e^{-\frac{1}{2}}
\end{equation*}
so $\int_1^\infty \exp\left(-\frac{x}{2}\right)$ exists, so $\int_1^\infty \exp \left(-\frac{x^2}{2}\right)$ exists, so $\int_{-\infty}^{-1} \exp \left(-\frac{x^2}{2}\right)$ exists, and thus $\int_{-\infty}^{+\infty} \exp \left(-\frac{x^2}{2}\right)$ exists
\end{proof}

However, we note that there is no way of writing $\int_{-\infty}^X \exp \left(-\frac{x^2}{2}\right) \, dx$ using $\log, \exp$, polynomials or trig functions.

\section{Further reading}

The two excellent books Spivak's \emph{Calculus}~\cite{Spivak} and J.~C.~Burkill's \emph{A First Course in Mathematical Analysis}~\cite{Burkill} both cover the course completely and should be in your college library\footnote{A quieter version of the JCR with shelves of books replacing the bar.}. Burkill's book is more condensed and Spivak's more leisurely. A completely unsuitable but interesting version of the standard analysis course is given by Berlinski's \emph{A Tour of the Calculus}~\cite{Berlinski} --- Spivak rewritten by Sterne with additional purple passages by the Ankh-Morpork tourist board. I have written \emph{A Companion to Analysis}~\cite{Korner} which covers this course at a higher level together with the next analysis course. It is available off the web but is unlikely to be as suitable for beginners as Spivak and Burkill. If you do download it, remember that you are under a moral obligation to send me an e-mail about any mistakes you find.

\begin{thebibliography}{9} 
\bibitem{Berlinski} D.~Berlinski \emph{A Tour of the Calculus} Mandarin Paperbacks 1997. 
\bibitem{Burkill} J.~C.~Burkill \emph{A First Course in Mathematical Analysis} CUP, 1962. 
\bibitem{Burn} R.~P.~Burn \emph{Numbers and Functions} CUP, 1992. 
\bibitem{Korner} T.~W.~K\"{o}rner \emph{A Companion to Analysis} for the moment available via my home page {\sf http://www.dpmms.cam.ac.uk/\textasciitilde twk/}\ . 
\bibitem{Spivak} M.~Spivak, \emph{Calculus} Addison-Wesley/Benjamin-Cummings, 1967. 
\end{thebibliography} 

\end{document}