RESULTS IN FIRST PART OF METHODS AND CALCULUS

T.W.KÖRNER

Definition 1. Let a_{n}, a ∈ ℝ^{k}. We say that a_{n} → a as n → ∞, if given ϵ > 0, we can find N(ϵ) such that ||a_{n} − a|| < ϵ for all n > N(ϵ).

Theorem 2. (i) If a_{n} → a and b_{n} → b in ℝ^{k} then a_{n} + b_{n} → a + b as n → ∞.

(ii) If a_{n} → a and b_{n} → b in ℝ (or ℂ) then a_{n}b_{n} → ab as n → ∞. (iii) If a_{n} → a as n → ∞ in ℝ (or ℂ) and a ≠ 0, a_{n} ≠ 0 [n = 1, 2, …]

then a_{n}^{−1} → a^{−1} as n → ∞.

There are many related definitions.

Definition 3. (i) If f : ℝ^{l} → ℝ^{k} we say that f(x) → a as x → y if given ϵ > 0 we can find a δ(ϵ) > 0 such that, whenever 0 < ||x − y|| < δ(ϵ) it follows that ||f(x) − a|| < ϵ.

(ii) Let a_{n} ∈ ℝ. We say that a_{n} → ∞ if, given any K we can find N(K) such that a_{n} > K for all n > N(K).

Axiom 4 (Fundamental Axiom of Analysis). If a_{1}, a_{2},… is an increasing sequence in ℝ and there exists an A ∈ ℝ such that a_{n} ≤ A for all n, then there exists an a ∈ ℝ such that a_{n} → a as n → ∞.

Definition 5. We say that the series a_{n} in ℝ^{k} converges to the sum a if

∑_{n=1}^{N} a_{n} → a as N → ∞.

We write

∑_{n=1}^{∞} a_{n} = a.

Lemma 6 (Absolute Convergence implies Convergence). If a_{n} ∈ ℝ^{k} and ∑_{n=1}^{∞} ||a_{n}|| converges then ∑_{n=1}^{∞} a_{n} converges.

Lemma 7 (Comparison Test). If a_{n}, b_{n} ∈ ℝ and 0 ≤ a_{n} ≤ b_{n} then whenever ∑_{n=1}^{∞} b_{n} converges ∑_{n=1}^{∞} a_{n} must converge.

Corollary 8 (Ratio Test). If a_{n} ∈ ℝ, 0 < a_{n} and a_{n+1}/a_{n} → l then (i) If l < 1 then ∑_{n=1}^{∞} a_{n} converges. (ii) If l > 1 then ∑_{n=1}^{∞} a_{n} diverges.

Example 9. If a_{2n} = 2^{−2n}, a_{2n+1} = 2^{−2n−2} then the ratio test fails but comparison with 2^{−n} show that the series a_{n} is convergent.

Lemma 10 (Integral Test). Suppose that f : [0, ∞) → [0, ∞) is a decreasing continuous function. Then if one of

∫_{0}^{N} f(x)dx and ∑_{n=0}^{N} f(n)

tends to a (finite) limit as N → ∞ so does the other.

Corollary 11. ∑_{n=1}^{∞} n^{−p} converges if and only if p > 1.

Note the failure of the ratio test for the series n^{−p}.

Example 12. If f(x) = 1 − cos(2πx) then ∫_{0}^{N} f(x)dx diverges and ∑_{n=0}^{N} f(n) converges as N → ∞.

Lemma 13 (Alternating Series Test). (Not in syllabus.) Suppose that a_{n} is a decreasing sequence of positive terms with a_{n} → 0 as n → ∞. Then ∑_{n=1}^{∞}(−1)^{n}a_{n} converges.

Example 14. ∑_{n=1}^{∞}(−1)^{n}n^{−p} converges for p > 0 but only converges absolutely for p > 1.

Theorem 15 (Rearrangement of Positive Series). Let σ : ℕ → ℕ be a bijection. If a_{n} ≥ 0 then, if the series a_{n} converges, so does the rearranged series a_{σ(n)} and

∑_{n=1}^{∞} a_{n} = ∑_{n=1}^{∞} a_{σ(n)}.

Corollary 16 (Rearrangement of Absolutely Convergent Series). (Not in syllabus.) Let σ : ℕ → ℕ be a bijection. If a_{n} ∈ ℝ^{k} then, if the series a_{n} is absolutely convergent, so is the rearranged series a_{σ(n)} and

∑_{n=1}^{∞} a_{n} = ∑_{n=1}^{∞} a_{σ(n)}.

Example 17. Let a_{2n−1} = n^{−1}, a_{2n} = −n^{−1} [n = 1, 2, …]. Then (by comparison with appropriate integrals)

∑_{n=1}^{kN} a_{2n−1} + ∑_{n=1}^{lN} a_{2n} = ∑_{n=lN+1}^{kN} n^{−1} → log(k/l)

as N → ∞ whenever k and l are integers with k ≥ l > 0.

Theorem 18. Let a_{n} ∈ ℂ.

(i)If ∑_{n=0}^{∞} a_{n}z_{0}^{n} converges then ∑_{n=0}^{∞} a_{n}z^{n} converges absolutely for all z ∈ ℂ with |z| < |z_{0}|.

(ii)If ∑_{n=0}^{∞} a_{n}z_{0}^{n} diverges then the sequence |a_{n}z^{n}| is unbounded for all z ∈ ℂ with |z| > |z_{0}|.

Theorem 19 (Radius of Convergence). (Proof next year.) Let a_{n} ∈ ℂ. Then, either ∑_{n=0}^{∞} a_{n}z^{n} converges for all z ∈ ℂ (and we say that the power series has radius of covergence infinity) or there exists an R with R ≥ 0 such that

(i) ∑_{n=0}^{∞} a_{n}z^{n} converges for all |z| < R,

(ii) ∑_{n=0}^{∞} a_{n}z^{n} diverges for all |z| > R. We call R the radius of convergence.

Example 20. (i) If a_{n} = 1/n! then R = ∞.

(ii) If a_{n} = n! then R = 0.

(iii) If a_{n} = r^{−n} then R = r.

(iv) If a_{4n} = 0, a_{4n+1} = 1, a_{4n+2} = 2, a_{4n+3} = 1 then R = 1.

(v) If a_{0} = 1, a_{n} = n^{p} then R = 1. If p < −1 then we have convergence of ∑_{n=0}^{∞} a_{n}z^{n} for |z| = 1, if p ≥ 0 we have divergence. If −1 ≤ p < 0 have convergence when z = −1 and divergence when z = 1.

Definition 21. We say that a function f : ℝ^{n} → ℝ^{m} is continuous at x ∈ ℝ^{n} if given ϵ > 0 we can find a δ(ϵ, x) > 0 such that, whenever ||h|| < δ(ϵ, x), we have ||f(x + h) − f(x)|| < ϵ.

Theorem 22. (i) If f : ℝ^{n} → ℝ^{m} is continuous at x ∈ ℝ^{n} and g : ℝ^{m} → ℝ^{p} is continuous at f(x) then the composed function g ∘ f (defined by g ∘ f(y) = g(f(y)) is continuous at x.

(ii) If F, G : ℝ^{n} → ℝ^{m} are continuous at x ∈ ℝ^{n} then so is F + G.

Definition 23. We say that a function f : ℝ^{n} → ℝ^{m} is differentiable at x ∈ ℝ^{n} with derivative the linear map α : ℝ^{n} → ℝ^{m} if given ϵ > 0 we can find a δ(ϵ, x) > 0 such that, whenever ||h|| < δ(ϵ, x), we have

||f(x + h) − f(x) − α(h)|| < ϵ||h||

We shall write α = (Df)(x). If we consider the special case m = n = 1 then (Df)(x) is a linear map ℝ → ℝ and so there exists a λ ∈ ℝ such that (Df)(x)h = λh for all h ∈ ℝ. We thus have

|f(x + h) − f(x) − λh|| < ϵ|h|

for |h| < δ(ϵ, x), and so

\frac{f(x + h) − f(x)}{h} → λ.

If we set λ = f′(x) we recover our old definition of differentiation. (Sometimes people write f′(x) = (Df)(x) in the general case, but we must then remember that f′(x) is a linear map.)

Lemma 24. Suppose that f : ℝ^{n} → ℝ^{m} is differentiable at x ∈ ℝ^{n} with derivative Df(x). If u is a vector in ℝ^{n} and v is a vector in ℝ^{m} then

g_{uv}(t) = v.f(x + tu)

defines a function g_{uv} : ℝ → ℝ which is differentiable at 0 with derivative v.(Df(x))(u).

Lemma 25. Suppose that u_{1}, u_{2},…, u_{n} is a basis for ℝ^{n} and v_{1}, v_{2},…, v_{m} is a basis for ℝ^{m}. If write f_{i,j}(x) = v_{i}.(Df(x))(u_{j}) then the linear map Df(x) : ℝ^{n} → ℝ^{m} has matrix (f_{i,j}(x)) with respect to the given bases.

There is an older and, in many circumstances, more convenient notation called the partial derivative. Suppose g is a well behaved real function of n real variables x_{1}, x_{2},…, x_{n}. Then we write

\frac{∂g}{∂x_{i}} = lim_{h→0} \frac{g(x_{1}, x_{2}, …, x_{i} + h, …, x_{n}) − g(x_{1}, x_{2}, …, x_{i}, …, x_{n})}{h}

for the derivative of g with respect to x_{i} when x_{1}, x_{2}, …, x_{i−1}, x_{i+1}, …, x_{n} are kept fixed. With this notation, if we write f_{j} for the jth component of the function f of the preceding lemma, we have

f_{j,i}(x) = \frac{∂f_{j}}{∂x_{i}}

However when reading material which uses partial derivatives (as in Classical Thermodynamics) it is important to be aware of possible ambiguities.

Example 26. Let f(x, y) = x + y. If we keep y fixed and allow x to

vary then \frac{∂f}{∂x} = 1, but if we keep x + y fixed and allow x to vary then

\frac{∂f}{∂x} = 0

Lemma 27. If α : ℝ^{n} → ℝ^{m} is linear there exists a constant A such that

||α(x)|| ≤ A||x||

for all x ∈ ℝ^{n}.

Theorem 28 (Chain Rule). If f : ℝ^{n} → ℝ^{m} is differentiable at x ∈ ℝ^{n} and g : ℝ^{m} → ℝ^{p} is differentiable at f(x) then the composed function g ∘ f (defined by g ∘ f(y) = g(f(y)) is differentiable at x with

D(g ∘ f)(x) = (D(g)(f(x))D(f)(x).

Corollary 29. (i) If f, g : ℝ^{n} → ℝ^{m} are differentiable at x ∈ ℝ^{n} then so is f + g and

D(f + g)(x) = D(f)(x) + D(g)(x).

(ii) If f, g : ℝ → ℝ are differentiable at x then so is their product and

\frac{d(f(x)g(x))}{dx} = g(x)\frac{d(f(x))}{dx} + f(x)\frac{d(g(x))}{dx}.

In the matrix notation of Lemma 25 the chain rule of Theorem 28 becomes

(g ∘ f)_{i,k}(x) = ∑_{j=1}^{m} g_{i,j}(f(x))f_{j,k}(x).

Still more briefly we may use the summation convention with i ranging from 1 to p, j from 1 to m and k from 1 to n to write

(g ∘ f)_{i,k}(x) = g_{i,j}(f(x))f_{j,k}(x).

If g is a real function of the real variables y_{1}, y_{2},…,y_{m} which are themselves functions of the real variables x_{1}, x_{2},…,x_{n} we recover the traditional form

\frac{∂g}{∂x_{k}} = \frac{∂g}{∂y_{1}} \frac{∂y_{1}}{∂x_{k}} + \frac{∂g}{∂y_{2}} \frac{∂y_{2}}{∂x_{k}} + … + \frac{∂g}{∂y_{m}} \frac{∂y_{m}}{∂x_{n}} .

If f : ℂ → ℂ then, just as in the real case we may define the derivative f′(z) by

f′(z) = lim_{h→0} \frac{f(z + h) − f(z)}{h}

where it exists (a ℂ → ℂ differentiable function is also called analytic). Since ℂ and ℝ are so similar those parts of the real theory which ought to go over to the complex case do.

Example 30. If P and Q are polynomials and Q has roots at ω_{1}, ω_{2}, …, ω_{n} then P/Q is differentiable at all z ∈ ℂ \ {ω_{1}, ω_{2}, …, ω_{n}}.

As the next theorem shows analytic functions are rather special.

Theorem 31 (Characterisation of Analytic Functions). Let f : ℂ → ℂ be given and write

f(x + iy) = u(x, y) + iv(x, y)

with x, y, u, v real. Let z_{0} = x_{0} + iy_{0}. The following statements are equivalent.

(i) f is differentiable as a function ℂ → ℂ at z_{0}.

(ii) f(z_{0} + h) = f(z_{0}) + λh + ϵ(h)|h| with λ ∈ ℂ and |ϵ(h)| → 0 as |h| → 0. (We have f′(z_{0}) = λ.)

(iii) Near z_{0}, f is the composition of translations, rotations, and (possibly zero) dilatations with an error which decreases faster than linear.

(iv) The function (u, v) : ℝ^{2} → ℝ^{2} is differentiable at (x_{0}, y_{0}) with derivative

D(u, v)(x_{0}, y_{0}) = lα

where l ≥ 0 and α ∈ SO(2) the special orthogonal group.

(v) (Cauchy Riemann Equations) The function (u, v) : ℝ^{2} → ℝ^{2} is differentiable at (x_{0}, y_{0}) and (at that point)

\frac{∂u}{∂x} = \frac{∂v}{∂y} , \frac{∂v}{∂x} = −\frac{∂u}{∂y}.

(vi) The function (u, v) : ℝ^{2} → ℝ^{2} is differentiable at (x_{0}, y_{0}) and (at that point)

\frac{∂u}{∂X} = \frac{∂v}{∂Y} , \frac{∂v}{∂X} = −\frac{∂u}{∂Y} .

for any orthogonal coordinate system (X, Y).

If f′(z_{0}) ≠ 0 then (iii) tells us that f is locally conformal (i.e. angle preserving).

Example 32. (i) The map z ↦ z^{2} is analytic but is not conformal at 0.

(ii) The mapping z ↦ z^{∗} is a reflection and nowhere analytic. It preserves the magnitude of angles but changes their sense.

Example 33. (This non-examinable example shows the need for a rigorous treatment of the calculus) Define f : ℚ → ℚ by

f(q) = q if q < 0 or q^{2} < 2,
f(q) = q − 3 otherwise.

Then, considered as a function ℚ → ℚ, f is continuous, yet f(1) = 1, f(2) = −1 and f(t) ≠ 0 for all t ∈ ℚ with 1 ≤ t ≤ 2. Further, considered as a function ℚ → ℚ, f is differentiable with derivative 1 everywhere, but f(1) > f(2) so f is not everywhere increasing.

Inspite of this, we shall be able next year to use the Fundamental Axiom (Axiom 4) to prove rigorously the following two fundamental theorems.

Theorem 34 (Intermeadiate Value Theorem). (Proof next year.) If f : [a, b] → ℝ is continuous and f(a) ≤ d ≤ f(b) then there exists a c with a ≤ c ≤ b and f(c) = d.

Theorem 35. (Proof next year.) If f : [a, b] → ℝ is continuous then f is bounded and attains its bounds. In other words we can find c_{1} and c_{2} with a ≤ c_{1}, c_{2} ≤ b and f(c_{1}) ≤ f(t) ≤ f(c_{2}) for all t ∈ [a, b].

We use Theorem 35 to give rigorous proofs of the results that follow.

Theorem 36 (Rolle’s Theorem). If f : [a, b] → ℝ is continuous, f is differentiable on (a, b) and f(a) = f(b) then there exists a c with a < c < b and f′(c) = 0.

Theorem 37 (Mean Value Theorem). If f : [a, b] → ℝ is continuous and f is differentiable on (a, b) then there exists a c with a < c < b and f(b) − f(a) = (b − a)f′(c).

Corollary 38. Suppose b > a and f : [a, b] → ℝ is continuous and f is differentiable on (a, b).

(i) If |f′(t)| ≤ M for all a < t < b then |f(a) − f(b)| ≤ M(b − a). (ii) If f′(t) = 0 for all a < t < b then f(b) = f(a). (iii) If f′(t) ≥ 0 for all a < t < b then f(b) ≥ f(a). (iv) If f′(t) > 0 for all a < t < b then f(b) > f(a).

Corollary 39. If f and g are differentiable functions (a, b) → ℝ with f′(t) = g′(t) for all a < t < b then we can find a constant c such that f(t) = g(t) + c for all a < t < b.

Example 40. If f(t) = t^{3} then f is strictly increasing yet f′(0) = 0.

The Mean Value Theorem does not go over unchanged to higher dimension.

Example 41. (i) Define f : ℝ → ℝ^{2} by f(t) = (cos(t), sin(t)). Then f(0) = f(2π) but Df(t) has matrix (− sin(t), cos(t)) and so is never zero.

(ii) Define g : ℂ → ℂ by g(z) = exp(iz). Then g(0) = g(2π) but g′(z) = i exp(iz) ≠ 0.

However there is a modified version which is just as useful.

Theorem 42 (Modified Mean Value Theorem). (i) Suppose f : ℝ^{n} → ℝ^{m} is differentiable with

||Df((1 − t)a + tb)h|| ≤ M||h||

for all h ∈ ℝ^{n} and all 0 ≤ t ≤ 1. Then

||f(b) − f(a)|| ≤ M||b − a||.

(ii) Suppose f : ℝ^{n} → ℝ^{m} is differentiable with Df(x) = 0 for all x with ||x − a|| < r.Then f(x) = f(a) for all x with ||x − a|| < r.

(iii) Suppose g : ℂ → ℂ is analytic with

|g′((1 − t)z_{1} + tz_{2})| ≤ M

for all 0 ≤ t ≤ 1. Then

|g(z_{2}) − g(z_{2})| ≤ M|z_{2} − z_{1})|.

(iv) Suppose g : ℂ → ℂ is analytic with g′(z) = 0 for all z ∈ ℂ with |z − w| < r. Then g(z) = g(w) for all z with |z − w| < r.

Lemma 43 (Operations Within Radius of Convergence). (Proof next year.)

(i) If ∑_{n=0}^{∞} a_{n}z^{n} has radius of convergence R and ∑_{n=0}^{∞} b_{n}z^{n} has radius of convergence S then, writing

c_{n} = ∑_{r=0}^{n} a_{n−r}b_{r},

∑_{n=0}^{∞} c_{n}z^{n} has radius of convergence T ≥ min(R, S) and

∑_{n=0}^{∞} a_{n}z^{n} ∑_{n=0}^{∞} b_{n}z^{n} = ∑_{n=0}^{∞} c_{n}z^{n}

for all |z| < min(R, S).

(ii) If ∑_{n=0}^{∞} a_{n}z^{n} has radius of convergence R then ∑_{n=1}^{∞} na_{n}z^{n−1} has the same radius of convergence. Futher ∑_{n=0}^{∞} a_{n}z^{n} is differentiable and

\frac{d}{dz} ∑_{n=0}^{∞} a_{n}z^{n} = ∑_{n=1}^{∞} na_{n}z^{n−1}

for all |z| < R.

Theorem 44. The infinite sum

exp(x) = ∑_{n=0}^{∞} \frac{x^{n}}{n!}

defines a function exp : ℝ → ℝ with the following properties. (i) exp is differentiable with exp′(x) = exp(x) for all x ∈ ℝ. (ii) exp(x + y) = exp(x) exp(y) for all x, y ∈ ℝ. (iii) exp(x) increases strictly from 0 to ∞ as x increases from −∞

to ∞.

Theorem 45. There exists a unique function log : (0, ∞) → ℝ such that log(exp(x)) = x for all x ∈ ℝ and exp(log(y)) = y for all y ∈ (0, ∞). The function log has the following properties.

(i) log is differentiable with log′(x) = 1/x for all x > 0.

(ii) log(xy) = log(x) + log(y) for all x, y > 0.

(iii) log(x) increases strictly from −∞ to ∞ as x increases from 0 to ∞.

Corollary 46. The additive group (ℝ, +) and the multiplicative group ((0, ∞), ×) are isomorphic.

Theorem 47. If we set x^{α} = exp(α log(x)) for x > 0, α ∈ ℝ then (i) x^{α}y^{α} = (xy)^{α}, (ii) x^{α}x^{β} = x^{(α+β)}, (iii) (x^{α})^{β} = x^{αβ}, (iv) x^{1} = x,

for all x, y > 0, α, β ∈ ℝ. Further, if α ∈ ℝ is fixed, x^{α} is differentiable with

\frac{dx^{α}}{dx} = αx^{α−1}.

Theorem 48. If we write

sin(x) = ∑_{n=0}^{∞} \frac{(−1)^{n}x^{2n+1}}{(2n + 1)!} ,

cos(x) = ∑_{n=0}^{∞} \frac{(−1)^{n}x^{2n}}{(2n)!} ,

then sin and cos are well defined differentiable functions ℝ → ℝ such that

(i) sin′(x) = cos(x) and cos′(x) = − sin(x),

(ii) sin(x+y) = sin(x) cos(y)+cos(x) sin(y), cos(x+y) = cos(x) cos(y)− sin(x) sin(y) and (cos(x))^{2} + (sin(x))^{2} = 1, for all x, y ∈ ℝ.

Moreover there exists a real number π such that 2 < π < 4 and (for x ∈ ℝ)

(iii) cos(x) ≥ 0 for 0 ≤ x < π/2, cos(π/2) = 0,

(iv) sin(x) = cos(x − π/2), cos(x + π) = − cos(x),

(v) If u, v ∈ ℝ and u^{2}+v^{2} = 1 then the equation (u, v) = (cos(θ), sin(θ)) has exactly one solution θ with 0 ≤ θ < 2π.

Corollary 49. If (x, y) ∈ ℝ^{2} and (x, y) ≠ (0, 0) then there is a unique r > 0 and a unique θ with 0 ≤ θ < 2π with x = r cos(θ), x = r sin(θ).

Theorem 50. We can extend the function exp in a consistent manner to a function ℂ → ℂ by setting

exp(z) = ∑_{n=0}^{∞} \frac{z^{n}}{n!}

With this definition, (i) exp is differentiable with exp′(z) = exp(z) for all z ∈ ℂ. (ii) exp(z_{1} + z_{2}) = exp(z_{1}) exp(z_{2}) for all z_{1}, z_{2} ∈ ℂ. (iii) If x and y are real, then

exp(x + iy) = exp(x)(cos(y) + i sin(y)).

Theorem 51. We can extend the functions cos and sin in a consistent manner to functions ℂ → ℂ by setting

sin(z) = \frac{exp(iz) − exp(−iz)}{2i},
cos(z) = \frac{exp(iz) + exp(−iz)}{2}.

With these definitions,

(i) cos and sin are differentiable with sin′(z) = cos(z) and cos′(z) = − sin(z),

(ii) sin(z+w) = sin(z) cos(w)+cos(z) sin(w), cos(z+w) = cos(z) cos(w)− sin(z) sin(w) and (cos(z))^{2} + (sin(z))^{2} = 1, for all z, w ∈ ℂ.

Lemma 52. If x is real, cosh(x) = cos(ix) and sinh(x) = −i sin(ix).

Theorem 53. (i) If w ∈ ℂ the equation z^{2} = w has a solution. However there is no continuous function f : ℂ → ℂ such that f(w)^{2} = w for all w ∈ ℂ.

(ii) The equation exp(z) = 0 has no solution. If w ≠ 0 then the set of solutions for exp(z) = w has the form

{log(|w|) + i(θ + 2nπ) : n ∈ ℤ}

for some real θ. However there is no continuous function g : ℂ\{0} → ℂ such that exp(g(w)) = w for all w ∈ ℂ \ {0}.

Definition 54. A continous map r : [a, b] → ℝ^{3} is called a curve in ℝ^{3}. If r(a) = r(b) we say that the curve is closed. We agree to identify (i.e. consider as the same) the two curves r_{1} : [a_{1}, b_{1}] → ℝ^{3} and r_{2} : [a_{2}, b_{2}] → ℝ^{3} if there exists a continuous bijective function γ : [a_{1}, b_{1}] → [a_{2}, b_{2}] with γ(a_{1}) = a_{2} and γ(b_{1}) = b_{2} such that r_{1}(t) = r_{2}(γ(t)).

Example 55. If the curves r_{j} : [0, 1] → ℝ^{3} are defined by

r_{1}(t) = (cos(2πt), sin(2πt), 0),
r_{2}(t) = (cos(2πt^{2}), sin(2πt^{2}), 0),
r_{3}(t) = (cos(4πt), sin(4πt), 0),

then r_{1} and r_{2} represent the same curve but r_{3} represents a different curve.

We say that r_{1} and r_{2} in the preceding definition are two parameterisations of the same curve.

Definition 56. If r : [0, l] → ℝ^{3} is continuous and

||\frac{r(s + δs) − r(s)}{δs}|| → 1

as δs → 0 we say that r is an arc length parameterisation of the curve.

Just as in the case of functions f : ℝ → ℝ where it is natural to identify the linear map Df(t) : ℝ → ℝ which takes h to f′(t)h with the real number f′(t) so, in the case of functions r : [0, l] → ℝ^{3}, it is natural to identify the linear map Dr(t) : [0, l] → ℝ^{3} which takes takes h to (ṙ_{1}(t)h, ṙ_{2}(t)h, ṙ_{3}(t)h) with the vector ṙ(t) = (ṙ_{1}(t), ṙ_{2}(t), ṙ_{3}(t)). We observe that (when r is well behaved)

||\frac{r(t + δt) − r(t)}{δt} − ṙ(t)|| → 0

as δt → 0.

Theorem 57. If r : [0, l] → ℝ^{3} is an arc length parameterisation of a well behaved curve then

ṙ(s) = t(s),

where t(s) is a unit vector defining the direction of the tangent. Either ṫ(s) = 0, or

ṫ(s) = \frac{n(s)}{ρ(s)},

where n(s) is a unit vector, perpendicular to t(s), defining the direction of the normal. The scalar ρ(s) > 0 is called the radius of curvature (not in the syllabus).

Theorem 58 (Taylor’s Theorem With Integral Remainder). Suppose that f : ℝ → ℝ is n + 1 times continuously differentiable. Then

f(x) = f(a)+f′(a)(x−a)+\frac{f′′(a)(x − a)^{2}}{2!} +\frac{f′′′(a)(x − a)^{3}}{3!} +…+\frac{f^{(n)}(a)(x − a)^{n}}{n!} +R_{n}(f, x),

where

R_{n}(f, x) = \frac{1}{n!} ∫_{a}^{x} f^{(n+1)}(t)(x − t)^{n}dt.

Corollary 59 (Local Taylor Theorem). If f : ℝ → ℝ is n times continuously differentiable then, given any ϵ > 0, there exists a δ(ϵ, a) such that

|f(x) − (f(a) + f′(a)(x − a) + \frac{f′′(a)(x − a)^{2}}{2!} … + \frac{f^{(n)}(a)(x − a)^{n}}{n!} )| < ϵ|x−a|^{n}

for all x with |x − a| < δ(ϵ, a).

Theorem 60 (Binomial Theorem). If α and and x are real and |x| < 1 then

(1 − x)^{α} = ∑_{n=0}^{∞} \frac{α(α − 1)(α − 2) … (α − (n − 1))}{n!}x^{n}.

If α is a positive integer there are only a finite number of non-zero terms; otherwise the sum diverges if |x| > 1.

Example 61 (Cauchy). Let

E(x) = exp(−1/x^{2}) for x ≠ 0,
E(0) = 0.

Then E is infinitely differentiable with

E^{(n)}(x) = Q_{n}(1/x) exp(−1/x^{2}) for x ≠ 0,
E^{(n)}(0) = 0,

where Q_{n} is a polynomial. The Taylor expansion

E(x) = ∑_{n=0}^{∞} \frac{E^{(n)}(0)}{n!}x^{n}

is only valid at the single point x = 0.

Next year we shall see that analytic functions (i.e. differentiable functions f : ℂ → ℂ) form such a restricted class that a Taylor expansion is always possible. (We shall make no essential use of this result so it serves mainly as a trailer for sensational results to come.)

Theorem 62 (Taylor’s Theorem For Analytic Functions). If f : {z ∈ ℂ : |z| < r} → ℂ is once complex differentiable it is infinitely complex differentiable and

f(z) = ∑_{n=0}^{∞} \frac{f^{(n)}(0)}{n!}z^{n}

for all |z| < r.

So far in this course we have been able to express the concepts in a direct geometric manner and then give computational rules for them involving coordinates. Our treatment of higher derivatives will, however, be purely computational. (This will be remedied in more advanced courses but the standard geometric approach requires a certain amount of multilinear algebra.)

In what follows we shall consider ℝ^{n} and ℝ^{m} each with a fixed basis and so fixed coordinate systems (x_{1}, x_{2}, …, x_{n}) and (y_{1}, y_{2}, …, y_{n}). Let f be a function ℝ^{n} → ℝ^{m}. Then, if 1 ≤ i ≤ m, f_{i} is a function ℝ^{n} → ℝ and (if f is well behaved) f_{i,j} is just another such function ℝ^{n} → ℝ [1 ≤ j ≤ n]. We can therefore form (f_{i,j})_{,k} [1 ≤ k ≤ n] and so on. To simplify notation we write

f_{i,jk} = (f_{i,j})_{,k}, f_{i,jkl} = (f_{i,jk})_{,l}, …

and so on [1 ≤ l ≤ n]. In partial derivative notation

f_{i,jk} = \frac{∂^{2}f_{i}}{∂x_{k}∂x_{j}} , f_{i,jkl} = \frac{∂^{3}f_{i}}{∂x_{l}∂x_{k}∂x_{j}} ,

Applying the Local Taylor Theorem of Corollary 59 to the function F : ℝ → ℝ given by F (t) = f_{i}(a + (x − a)t) we obtain the following multidimensional version.

Theorem 63 (Local Multidimensional Taylor Theorem). Let f be a function ℝ^{n} → ℝ^{m}. If f is sufficiently well behaved (more exactly if f has continuous partial derivatives of sufficiently high order) then, given any ϵ > 0, there exists a δ(ϵ, a) such that

||f_{i}(x) − (f_{i}(a) + ∑_{1≤j≤n} f_{i,j}(a)(x_{j} − a_{j}) + ∑_{1≤j,k≤n} \frac{f_{i,jk}(a)(x_{j} − a_{j})(x_{k} − a_{k})}{2!} + …

+ ∑_{1≤j,k,...,p≤n} \frac{f_{i,jk...p}(a)(x_{j} − a_{j})(x_{k} − a_{k}) … (x_{p} − a_{p})}{N!})|| < ϵ||x − a||^{N}.

By a careful exploitation of these ideas (which we leave to next year) it is possible to prove the following useful result.

Theorem 64 (Partial Differentiation Commutes). Let f be a function ℝ → ℝ^{m}. If f is sufficiently well behaved (more exactly if f has continuous second partial derivatives) then f_{,jk} = f_{,kj}.

In other words

\frac{∂^{2}f}{∂x_{k}∂x_{j}} = \frac{∂^{2}f}{∂x_{j}∂x_{k}} .

Applied to the function of Theorem 63 the result shows that f_{i,jk} = f_{i,kj} and that the higher order terms have similar symmetries. Note that some condition on the behaviour of f is required in Theorem 64 since it is possible to construct pathological functions for which f_{,jk} and f_{,kj} exist but are not equal.

Combining Theorem 62 which says that analytic functions are well behaved, with Theorem 64 which says that, for well behaved functions, the partial derivatives commute and with the Cauchy Riemann equations of Theorem 31 we obtain the following result which turns out to be very important for later work.

Theorem 65. If f : ℂ → ℂ is a well behaved analytic function and we write f(x + iy) = u(x, y) + iv(x, y) as in Theorem 31 then u and v both satisfy Laplace’s equation

\frac{∂^{2}ϕ}{∂x^{2}} + \frac{∂^{2}ϕ}{∂y^{2}} = 0.

Functions which satisfy Laplace’s equation are called harmonic. In the second half of this course you will see that the converse to Theorem 65 holds and every harmonic function is (at least locally) the real part of an analytic function.

We now return to the local Taylor expansion of Theorem 63 in the special but important case when m = 1 concentrating particularly on the cases when n ≤ 3 which are easiest to visualise. If f : ℝ^{n} → ℝ is well behaved Theorem 63 tells us that

f(x) = f(a)+ ∑_{1≤j≤n} f_{,j}(a)(x_{j}−a_{j})+ ∑_{1≤j,k≤n} \frac{f_{,jk}(a)(x_{j} − a_{j})(x_{k} − a_{k})}{2!}+error,

where the error term decreases faster than ||x − a||^{2} as x approaches a. Bearing in mind the symmetry of the second derivative, we may write this in matrix terms as

f(a + h) = f(a) + b^{T} h + \frac{1}{2}h^{T} Bh + error, (∗′)

where h is a column vector, b is a column vector with b_{j} = f_{,j}(a) and B is an n × n symmetric matrix with b_{jk} = f_{,jk}(a). We call B the Hessian matrix. We give b the name grad(f), ∇f, or ∇f(a) so that (∗) becomes

f(a + h) = (f(a) + ∇f(a)^{T} h + \frac{1}{2}h^{T} Bh + error, (∗′′)

Alternatively we may write the equation in terms of linear maps as

f(a + h) = f(a) + (Df(a))(h) + \frac{1}{2}h.βh + error, (∗∗)

where the dot denotes inner product and β is the linear map with matrix B. Combining the ideas of (∗′′) and (∗∗) gives yet annother form of the equation.

f(a + h) = f(a) + ∇f(a).(h) + \frac{1}{2}h.βh + error. (∗)

To help follow the rest of this discussion the reader should draw contour lines (that is lines in ℝ^{2} on which f is constant) for f when n = 2 and imagine contour surfaces (that is surfaces in ℝ^{3} on which f is constant) for f when n = 3. Observe that (∗) tells us that

f(a + δa) − f(a) = ∇f(a).δa + error,

or, still more briefly

δf = ∇f.δa + error,

where the error decreases faster than linearly. If we move along a line (or surface) on which f is constant then δf = 0 so ∇f must be perpendicular to contour lines (or surfaces). Working in this context it is clear that ∇f is a vector in the direction that f changes most rapidly of length proportional to the rate of change of f in that direction^{1}

So long as ∇f ≠ 0 the linear term ∇f.δh dominates all the other non-constant terms in (∗) (so, in particular, we cannot have a maximum or a minimum at such a point). At, so called, stationary points ∇f = 0 and (∗) becomes

f(a + h) − f(a) = \frac{1}{2}h.βh + error. (∗)

If we consider the case when n = 2 and write (∗) in coordinate form we get

f(a_{1} + h_{1}, a_{2} + h_{2}) − f(a_{1}, a_{2}) = \frac{1}{2}(f_{,11}h_{1}^{2} + 2f_{,12}h_{1}h_{2} + f_{,22}h_{2}^{2}) + error,

with the error decreasing faster than quadratically. Using the ideas of the previous course we see that

(i) If f_{,11} > 0 and det B = f_{,11}f_{,22} − f_{,12}^{2} > 0 then the Hessian

B = ( \begin{matrix}f_{,11} & f_{,12} \\ f_{,21} & f_{,22} \end{matrix} )

is positive definite and a is a minimum.

(ii) If f_{,11} < 0 and det B < 0 then the Hessian is negative definite and a is a maximum.

(iii) If f_{,11} and det B are non-zero and of opposite signs then a is not a maximum nor a minimum. (We have a saddle point.)

(iv) In all other cases the behaviour depends on higher order terms. (But see part (ii) of Remark 66 below.) No new phenomena emerge in higher dimensions (n ≥ 3) but the calculations become a bit more complicated.

We make the following remarks which are non-examinable and will be proved by hand waving.

1WARNING The geometric introduction of ∇f above depends on using a particular inner product. ∇f behaves as a vector so long as we confine ourselves to orthogonal changes of coordinates. If we change our scales of measurement then it behaves in an unexpected manner. The Pure Mathematician explains this by saying that ∇f is indeed a vector but lives in a dual space and the Applied Mathematician explains this by saying that ∇f is indeed a vector but of contravariant type. Fortunately the problem does not arrise until Part II, and, possibly, not even then, so you may snopake this footnote out and forget it.

Remark 66. (i) If f : ℝ → ℝ is a well behaved function and ϵ > 0 there exists a well behaved perturbation f̃: ℝ → ℝ such that |f(t) − f̃(t)| < ϵ for all t ∈ ℝ and all the stationary points of f̃ are maxima and minima.

(ii) If g : ℝ^{2} → ℝ is a well behaved function and ϵ > 0 there exists a well behaved perturbation g̃ : ℝ^{2} → ℝ such that |g(t) − g̃(t)| < ϵ for all t ∈ ℝ^{2} and, at each stationary point the matrix

( \begin{matrix}g̃_{,11} & g̃_{,12} \\ g̃_{,21} & g̃_{,22} \end{matrix} )

is invertible and g̃_{,11} ≠ 0.

The following theorem (which is not in the syllabus) indicates why saddle points cannot be perturbed away.

Theorem 67 (Lakes, Peaks and Passes). Consider the sphere

S_{2} = {x ∈ ℝ^{3} : ||x|| = 1}.

If f : S_{2} → ℝ is well behaved with L minima, P maxima and S saddle points then P − S + L = 2.

More generally the number P − S + L is a ‘topological invariant’ for the associated surface.