This post is an excuse for me to share my notes on some very interesting topics of real analysis. The Basel problem is the problem of answering the following two questions about the inifinite series defined as: \(1+\frac{1}{2^2}+\frac{1}{3^2}+\frac{1}{4^2}+\dotsm\)
- Does this series converge?
- If yes, then to what value?
We will see that the answers to the two questions are yes and \(\pi^2/6\). But that's not all, we will see that this problem is a great entry point into the origins of real analysis.
1 Showing that the series converges
If we look at this series without giving much thought, it is easy to convince ourselves that the series converges because the terms being added to the series get progressively smaller, so that changes to the sum of the series stop after some point. But, there's another series with the same property of terms getting smaller and smaller that does not converge; the harmonic series: \(1+\frac{1}{2}+\frac{1}{3}+\frac{1}{4}+\dotsm\)
I really like this example because it can bring out errors we make by assuming things to be overly simplistic, this is a proverbial edge case in the program of our mind. Now that we know we need a little more sophistication, let's build the tools that we require. We first need to define what the sum of such an infinite series means.
Definition 1 (Sum of infinite series). Let \(a_n\) be the \(n\)-th term of the series, i.e. the series is \(a_1+a_2+\dotsm+a_n+\dotsm\), then define the \(n\)-th partial sum as \(s_n=\sum_{k=1}^na_k\). We look at the sequence \((s_1,s_2,\dotsc)\), if this sequence converges then we say that the sum of the series is \(\displaystyle S=\lim_{n\to\infty} s_n\). If it does not converge then we leave the sum undefined.
The above definition hints to us that we need to look at the limit of the partial sums to answer our question. So, we need to analyse \(s_n=\sum_{k=1}^na_k\) with \(a_k=\frac{1}{k^2}\). We will then modify this as follows.
$$\begin{aligned} \frac{1}{k^2}&=\frac{1}{k}\cdot\frac{1}{k}\\ &<\frac{1}{k-1}\cdot\frac{1}{k}\\ &=\frac{1}{k-1}-\frac{1}{k}\\ \implies& s_n\leq 1+\sum_{k=2}^n \frac{1}{k-1}-\frac{1}{k} \end{aligned}$$
Let's denote the sum \(1+\sum_{k=2}^n \frac{1}{k-1}-\frac{1}{k}\) by \(\sigma_n\). Since \(s_n\leq\sigma_n\), we know using the limit laws that \[\lim_{n\to\infty}s_n\leq\lim_{n\to\infty}\sigma_n.\] The sum \(\sigma_n\) is finite, and is equal to just a number. So, we can easily re-arrange its terms and see that
$$\begin{aligned} \sigma_n&=1+\sum_{k=2}^n\left(\frac{1}{k-1}-\frac{1}{k}\right)\\ &=1+\left( 1-\frac{1}{2}\right)+\left( \frac{1}{2}-\frac{1}{3}\right)+\left(\frac{1}{3}-\frac{1}{4}\right)+\dotsm+\left(\frac{1}{n-1}-\frac{1}{n}\right)\\ &=1+1+\left(\frac{1}{2}-\frac{1}{2}\right)+\left(\frac{1}{3}-\frac{1}{3}\right)+\dotsm+\left(\frac{1}{n-1}-\frac{1}{n-1}\right)-\frac{1}{n}\\ &=2-\frac{1}{n} \end{aligned} $$
We will now evaluate the infinite sum whose partial sums are \(\sigma_n\). The infinite sum is \(\displaystyle\lim_{n\to\infty}\sigma_n=2\). Therefore, we know that \(1+\frac{1}{2^2}+\frac{1}{3^2}+\dotsm\leq 2\). Therefore, the series converges to a sum less than two.
\(\square\)
2 The Fourier Series
The solution of the Basel problem can be found in several ways. Here we will look at only one of those ways. This solutions is ingenious, and uses the Fourier series. We need to analyze the periodic function \(f(x)=x\) over the interval \([-\pi,\pi)\), and with a period of \(2\pi\). What this means is that \(f(x+2n\pi)=f(x)\), so the function is completely defined by defining it over any interval of length \(2\pi\). We need to compute the Fourier series of \(f\). The Fourier coefficients \(\hat{f}(n)\) are computed using the formula \[\hat{f}(n)=\frac{1}{\sqrt{2\pi}}\int_{-\pi}^\pi f(x)e^{-inx}dx\] and then the Fourier series is defined as \(\frac{1}{\sqrt{2\pi}}\sum_{n=-N}^N\hat{f}(n)e^{inx}\), where \(i=\sqrt{-1}\). Fejér proved that this series converges to the original function \(f\) uniformly as \(N\to\infty\). (Fejér didn't prove it just for the specific function \(f(x)=x\), instead he proved it for a general class of functions in which our function lies.)
So, let's compute the Fourier coefficients \(\hat{f}(n)\). We will need to use integration by parts to compute the integral \[\int_{-\pi}^\pi xe^{-inx}dx.\] I will encourage you to do it yourself. Once you integrate by parts, you have to use the identity \(e^{i\theta}=\cos\theta+i\sin\theta\) along with the facts that \(\cos(-\theta)=\cos\theta\) and \(\sin(-\theta)=-\sin\theta\). This will give us for \(n\neq 0\) \[\hat{f}(n)=\frac{i\sqrt{2\pi}(-1)^n}{n}\] and \(\hat{f}(0)=0\).
When we create the Cesàro
sums of the Fourier series
we have computed above, we see the following approximation of our function. You
can play with the graph here.
Thus far, we haven't seen our infinite series anywhere. It'll make its entrance shortly, but we need to discuss about something else to prepare for its arrival.
3 Connections to Hilbert Spaces
We call the functions \(\frac{1}{\sqrt{2\pi}}e^{inx}\) trigonometric. The trigonometric series is a special class of orthogonal system. This allows Fourier series to be naturally extended to the context of Hilbert spaces. A Hilbert space is a vector space, that has an inner product, and is complete with respect to the metric induced by that inner product. A familiar inner product is the dot product in the \(\mathbf{R}^n\) vector space.
What this means is that we can form a vector space of functions; we can endow it with an inner product; and we can ensure that it is complete to give rise to a Hilbert space whose elements are functions, and then use all the theories of the linear algebra over the functions as if they were vectors. All periodic functions \(g\) over a fixed interval that are also square integrable (this means that \(\int|g(x)|^2<\infty\)) do in fact form a Hilbert space.
The key property that we are interested in is that the inner product also induces a norm defined as \(\lVert v\rVert^2=\langle v,v\rangle\), where \(\langle \cdot,\cdot\rangle\) represents the inner product. There is another way to compute this norm, by using the co-ordinates of the vector in terms of a orthonormal basis and using the pythagoras theorem. That is, if \(v=v_1e_1+v_2e_2+\dotsm\) and \(e_i\perp e_j\) (which means \(\langle e_i,e_j\rangle=0\)) for \(i\neq j\) and \(\lVert e_i\rVert=1\) then \(\lVert v\rVert^2=\sum_{i}v_i^2\).
Example. Let \(v=e_1+2e_2\), \(v\in\mathbf{R}^2\), then $$ \begin{aligned} \lVert v\rVert^2&=\langle v,v\rangle\\ &=\langle e_1+2e_2,e_2+2e_2\rangle\\ &=\langle e_1,e_1\rangle+\langle e_1,2e_2\rangle+\langle 2e_2,e_1\rangle+\langle 2e_2,2e_2\rangle\\ &=1+0+0+4=5 \end{aligned} $$ which is the same as \(1^2+2^2=5\).
Remark. With the standard basis, we will have \(e_1=\begin{bmatrix}1\\0\end{bmatrix}\) and \(e_2=\begin{bmatrix}0\\1\end{bmatrix}\) so that \(v=\begin{bmatrix}1\\2\end{bmatrix}\) and \(\langle v,v\rangle=\begin{bmatrix}1&2\end{bmatrix}\begin{bmatrix}1\\2\end{bmatrix}=5\), which by definition of the matrix product is the same as \(1^2+2^2\).
Now that we have had a little experience with this, it is time to introduce more concepts of the Hilbert space of the function. First, the period of the functions is \(2\pi\) in our case; second the inner product is defined as follows \[\langle v,v\rangle=\int_{-\pi}^\pi f(x)\overline{g(x)}dx\] where \(\overline{g(x)}\) represents the complex conjugate of \(g(x)\); third the basis of this vector space is \(e_n=\frac{1}{\sqrt{2\pi}}e^{inx}\).
Remark. Remember that functions are vectors now, and the basis is also made of vectors. Therefore, the basis is also a function of \(x\). Moreover, this is just one of the possible basis of the space we are interested in.
With all the notation in place, we can recognize that when writing the Fourier series of a function, we are actually writing it in terms of the coordinates of these basis functions.
$$ \begin{aligned} f&=\lim_{N\to\infty}\sum_{n=-N}^N\hat{f}(n)e_n\\ &=\sum_{n=-\infty}^\infty\langle f,e_n\rangle e_n \end{aligned} $$
4 The Basel solution
This is the final step; most of our computation and theory is done already. We will now compute the norm of the function \(f\), that we have been analyzing for a while, in two ways. \[ \begin{aligned} \lVert f\rVert^2=\langle f,f\rangle&=\int_{-\pi}^\pi x^2dx\\ &=\frac{2\pi^3}{3}\\ &=\sum_{n=-\infty}^\infty |\hat{f}(n)|^2\\ &=2\sum_{n=1}^\infty |\hat{f}(n)|^2 \end{aligned} \] Now, let's compute the absolute value of \(\hat{f}(n)\) using the fact that for any complex number \(z\), \(|z|=\sqrt{z\overline{z}}\). \[ \begin{aligned} |\hat{f}(n)|^2&=\frac{i\sqrt{2\pi}(-1)^n}{n}\cdot\frac{-i\sqrt{2\pi}(-1)^n}{n}\\ &=\frac{-i^22\pi(-1)^{2n}}{n^2}\\ &=\frac{2\pi}{n^2} \end{aligned} \]
Putting this back into the above equation of the norm gives us \[ \begin{aligned} \frac{2\pi^3}{3}&=2\sum_{n=1}^\infty\frac{2\pi}{n^2}\\ \implies \frac{\pi^3}{3}&=2\pi\sum_{n=1}^\infty\frac{1}{n^2}\\ \implies \frac{\pi^2}{6}&=\sum_{n=1}^\infty\frac{1}{n^2} \end{aligned} \]
\(\square\)
As a sanity check: \(\pi^2/6=1.6449\) which is indeed less than \(2\).5 The Connection to Real Analysis
I was just trying to prove that the Basel sum converges and evaluates to \(\pi^2/6\), but I couldn't convince myself about the convergence of the Fourier series. Trying to find answers to these questions led me towards the Weierstrass approximation theorem for polynomials, the Weierstrass approximation theorem for trigonometric polynomials, the theory of convolutions, the theory of Hilbert spaces, and some amazing proofs by Fejér, Euler, Dirichlet, and others. Further, I learned that the initial seeds of real analysis were sown by mathematicians trying to answer questions about the convergence of the Fourier series like "what is the type of convergence", "what are the conditions under which the convergence happens", etc. I have elaborated more on these in my notes: full solution of Basel problem, the Weierstrass theorem and the proof of the convergence of Fourier series.
Conclusion
Periodic functions can be seen as functions defined on the circle, and the input is the angle. If you color code the values of the function, you can think of this as a coloring of the circle, where the usual functions are colorings of the real number line. Or, you could think of it as wrapping the graph around a circle. And the trigonometric functions (\(e^{inx}\)) form a basis for all such functions. This idea can be extended to functions defined on a sphere giving rise to spherical harmonics. Spherical harmonics are a cornerstone for analyzing the group of 3D rotations, and because of the connections of 3D rotations and quantum mechanics, they also play an important role in modeling atomic interactions. My next goal will be to learn these and write about them. Stay tuned!
Appendix: A Note on the Choice of the Inner Product
Usually in \(\mathbf{R}^n\) with \(n\) a finite natural number, we have the familiar inner product (the dot product) which can be written as \[u\cdot v=\sum_{i=0}^nu_iv_i.\] When we have the complex numbers as the underlying field over which we have the vector space, then we have a slight modification of this as \[u\cdot v=\sum_{i=0}^nu_i\overline{v_i}.\] This is to ensure that the inner product follows all the axioms of an inner product (try to think of an example of vectors that will defy one of the axioms of the inner product when the coordinates of the vectors consist of complex numbers). The inner product in the \(L^2\) space is just a generalization of this. \(\sum\) becomes \(\int\) to give \[\langle u,v\rangle=\int u\overline{v}.\] In particular, \(u\) and \(v\) are functions of \(x\) so we make that explicit, and we use more familiar letter for denoting functions therefore we get back our familiar inner product \[\int f(x)\overline{g(x)}dx.\] The reason we have \(-\pi\) and \(\pi\) as the limits of our integral is that we need the inner product to be finite for it to be meaningful, and this plays very well with our situation where the functions are periodic and thus we can assume the inner product over the period contains just as much information as an inner product over a larger domain would.
As a final note, observe that we can think of vectors as functions as well and that is very beneficial in programming. A vector \(v\in\mathbf{R}^3\) is a function \(v:\{1,2,3\}\to\mathbf{R}\), it assigns a real number to the natural numbers. This allows us to write the dot product as \[u\cdot v=\sum_{i=0}^nu(i)\overline{v(i)}\] which is much closer to the inner product between functions. With functions as well, we take the sum of pointwise products.
In programming, arrays replace vectors, acting as functions from natural numbers to values. These are discrete functions with arbitrary mappings. Functions can be represented as input-output pairs in a table (relational view). For discrete functions, this allows caching computed results in arrays, forming the basis of dynamic programming. When you think of vectors or arrays as functions, it allows you to think composably or recursively while programming, and helps imagine the full pipeline of your code as a sequence blocks where each block is a function that takes an input and produces an output. Such a diagram is what a flowchart is.