surprising connections between number theory and physicsIn a textbook of 1946, Erwin Schrödinger was working out the behaviour of an ideal quantum gas. He needed two integrals; he expanded them as infinite series; the series converged more and more reluctantly as a certain parameter approached 1. He called that parameter ζ. Then, having said that he was not going to pursue the matter, he added a parenthesis: "Convergence becomes poorer as ζ increases but actually subsists till ζ = 1. (For ζ = 1, by the way, the series represent, apart from simple factors, the Riemannian ζ function of argument 3/2 and 5/2 respectively, which can be found in tables.)"
The same Greek letter is doing two entirely unrelated jobs in that sentence, which is confusing, and which is also the whole point. Six pages later Schrödinger needed three more definite integrals and wrote them straight down as ½ζ(2) = π2/12, then 7π4/120, then 31π6/252, observing only that "the expressions in π come from a formula" – Euler's, which ties the zeta function at even numbers to the Bernoulli numbers. No comment. Why would there be a comment? He needed a number, and the number was in a table. If you have never met the Riemann zeta function, here is the one thing worth knowing about it. Euler noticed that adding up 1 + 1/2s + 1/3s + 1/4s + … gives exactly the same answer as multiplying together one factor for each prime number – so a function defined without any mention of primes turns out to have all of them hidden inside it. In 1859 Riemann showed that the fine detail of how the primes are distributed – how many there are up to a given size, how they thin out, where they bunch – is controlled by the points at which this function takes the value zero. Where exactly those zeros lie is the Riemann hypothesis, unresolved ever since, and generally considered the most important open problem in mathematics. The zeta function is not just another piece of mathematical furniture. It's the thing that "knows" where all the prime numbers are, simultaneously. And there it sits in a book about gases, twice in seven pages, as a lookup table. By the way. Which can be found in tables. There are two ways to take that. The first is that it means nothing at all: these integrals turn up everywhere in physics, ζ(s) is just a compact name for a particular sum, and Schrödinger was entirely right to be unbothered by it. The second is that it means something – that when the primes' own function appears in the thermodynamics of a gas of indistinguishable particles, that is a clue and not a coincidence. This site is what happens if you take the second possibility seriously for twenty-five years. It aims to document as much as possible of the research which in some way links number theory and physics. Although there have been a few conferences and subsequently published proceedings on the topic, these have only been able to touch on a small part of the body of work that has gone on. What is collected here should be of interest to both number theorists and physicists: in recent times we have seen, somewhat unexpectedly, number theory being applied by physicists to solve physical problems and, perhaps even more unexpectedly, techniques developed by physicists applied to problems in number theory. Material relevant to all such developments is archived in the sections linked from the upper part of the front page. I cannot claim that the second reading is the right one. But what can be done is to gather the evidence in one place, so that the pattern – if it is a pattern – has somewhere to become visible. Schrödinger wrote "by the way". Taken as a whole, the archive is a long, patient argument with that parenthesis. Around that central question, the archive has accumulated a considerable amount of more obliquely related or "secondary" material, organised into categories linked from the lower part of the front page. Probability and statistics, for example, are not in themselves branches of physics. But they were developed to describe events, measurements and fluctuations in the changing physical world. Their applicability to something as unphysical and unchanging as the prime numbers has long been regarded as curious, remarkable and surprising. To treat the occurrence of a prime as a kind of random event is to apply to the pure, eternal world of number a way of thinking born in the physical world. Hardy and Littlewood commented in [HL] that "Probability is not a notion of pure mathematics, but of physics or philosophy." Still, as T. Gowers has observed in [Go] "Although the prime numbers are rigidly determined, they somehow feel like experimental data." The section concerning "fractality" is perhaps less directly physics-related, but a significant part of the content is the work of physicists. The archive has naturally expanded in some directions which lie outside its originally envisaged scope. Some of the sections (Bernoulli numbers, golden mean,...) are not particularly physics-related, but contain unorthodox, innovative, speculative, or just curious approaches to number theoretic issues. As well as links and archived electronic articles relating to in-depth material, I have included some introductory number theory resources (follow the tutorial link from the main page). These are for the benefit of students, interested amateurs who wish to educate themselves in these matters, and research physicists whose work has begun to reveal some unexpected connection with number theory and who therefore need to quickly learn the basics. My own attraction to the subject matter stems from a vague, but deeply-rooted feeling that these newly emerging connections are beginning to reveal something quite extraordinary and unexpected about the very nature of the number system, something which has been hitherto inaccessible. I am overwhelmed with a sense of mystery when I browse the contents of this archive. It's not just me – the prime numbers unto themselves have inspired some remarkable quotations from those who have studied them (collected here). The unexpected connections with physics, I feel, significantly compound this widely acknowledged mystery. Although I am fully aware that my understanding is limited and possibly misguided to some extent, my intuition and enthusiasm compel me to dedicate a significant amount of time to this project, despite the lack of any financial support. It is my hope that this site will catalyse and accelerate the cross-fertilisation of ideas between number theory and physics, as well as inspiring others to use the Web in a similar spirit. The future of mathematics could be essentially a single, enormous, web of hypertext documents, potentially accessible to the whole of humanity. Putting aside organisational difficulties, this project happily looks forward to its possible destiny as a small piece of this great web of knowledge. Contributions are welcomed. I am always happy to see this project expand and
would like to involve as many other people as possible.
Here's an excellent survey article: D. Schumayer and D.A.W. Hutchinson, "Physics of the Riemann Hypothesis" (preprint 01/2011), published as Reviews of Modern Physics 83 (2011) 307–330.
If you read one thing before the rest of this page, read that.
Below is a fairly brief overview of some of the more significant content of the archive. A word on status, since this survey deliberately mixes registers. Some of the work below – Julia, Bost–Connes, Berry–Keating, Deninger, Connes, Keating–Snaith – is mainstream published research. Other material, flagged where it appears, sits outside the accepted literature. It is included because this archive has always erred towards inclusion, but a reader arriving here from a search engine deserves to be told which is which rather than left to guess. partition functions and the free Riemann gasBernard Julia of the Laboratoire de Physique Théorique de l'École Normale Supérieure in Paris has reinterpreted the (pure mathematical) Riemann zeta function as a (thermodynamic) partition function by defining an abstract numerical "gas" using the prime numbers ([J]). The free Riemann gas is a surprisingly natural concept, and its partition function is identical to the zeta function. In statistical mechanics, the partition function is the fundamental mathematical object of study; in the analytic theory of the distribution of primes, the zeta function is the fundamental object. Hence this unorthodox interpretation of the zeta function as a partition function points to a possible link of fundamental significance between the distribution of primes and this branch of physics. Julia has further linked the pole of the zeta function at s = 1 with the physical phenomenon known as a Hagedorn catastrophe which occurs when a system reaches a critical Hagedorn temperature ([H1-3]). Physical phenomena such as Bose condensation and ladders of fermion models are also indirectly linked to the zeta function by Julia in this paper. Donald Spector has informed me that he independently and simultaneously recognised the connection with partition functions, the Hagedorn temperature, etc. ([Sp]). His own papers draw on the theory of supersymmetry and suggest several intriguing parallels between phenomena in physics and multiplicative number theory. Bost and Connes' groundbreaking paper [BC] was partly inspired by the work of Julia, and also involves a model in which the Riemann zeta function is interpreted as a partition function. In this model, the pole of zeta at 1 is understood in terms of a spontaneous breaking of symmetry. D. Fivel has independently interpreted the Riemann zeta function as a partition function in an entirely different context – that of quantum entanglement ([F]). Andreas Knauf and others have been studying spin chains (essentially one-dimensional lattice models) and in many cases the relevant partition functions involve the Riemann zeta function and its derivative in various combinations. In a 1999 lecture in Budapest ([W1]), Marek Wolf, a Wroclaw University physicist, presented a variant on Julia's idea, defining another kind of "prime gas", this time where the partition function resembles that of a quantum harmonic oscillator. Wolf's gas differs from Julia's in that the gaps between the primes are taken to be the fundamental quantities, rather than primes themselves. A summary of Wolf's lecture can be found here and provides a good review of much work done in this area. All of this work (and more) is documented in the section of the number theory and physics archive dedicated to statistical mechanics. Whereas everything mentioned thus far relates to multiplicative number theory, there is an additional body of work going back to the 1940's which uses the methods of statistical mechanics to investigate the problems of additive number theory, in particular the partitioning of integers. the spectral interpretation and random matricesAround 1912–14, about half a century after Riemann published his famous hypothesis, Pólya suggested, in conversation with Landau, that it could be proved if the nontrivial (complex) zeros of zeta could be shown to correspond directly to the spectrum of eigenvalues of some self-adjoint operator on a Hilbert space. Evidence for the validity of this "spectral interpretation" of the Riemann zeta function has since come from two sources:
Two things have strengthened this considerably since the above was written. The numerical evidence went much further – Odlyzko computed zeros in the neighbourhood of the 1022-nd [O22], where the agreement with the Gaussian Unitary Ensemble is extraordinarily close, and where the small remaining deviations are themselves accounted for by the lower-order terms predicted by Bogomolny and Keating [BoK]. More importantly, the GUE hypothesis is no longer supported only by computation: Rudnick and Sarnak proved that the n-level correlations of the zeros agree with the random matrix prediction for test functions of restricted support [RS], and Katz and Sarnak established the underlying symmetry philosophy outright in the function-field setting [KaS], where the analogue of the Riemann hypothesis is a theorem rather than a conjecture. Random matrix theory has also stopped being merely descriptive. By modelling the zeta function on the characteristic polynomials of random unitary matrices, Keating and Snaith were able to predict the moments of zeta on the critical line [KSn] – quantities that decades of purely number-theoretic work had not reached – and this was later extended into a general conjectural framework for moments of L-functions [CF]. Whatever one makes of the spectral interpretation, this is the point at which the physics stopped being an analogy and started making arithmetic predictions. The Hilbert–Pólya conjecture, as it has become known, still seems to be the most promising approach to proving the Riemann hypothesis. Hilbert's name attached itself to the conjecture later, and it is worth saying plainly that no written record of Hilbert making the suggestion is known: the whole documented basis is a letter Pólya wrote to Odlyzko in 1982, recalling the conversation with Landau some seventy years after it happened [Po]. In [J] Julia acknowledges the celebrated work of Sir Michael Berry and Jon Keating ([BK]) of Bristol University, who have also been looking at the Riemann zeta function from a dynamical viewpoint. Theirs is certainly the most widely known of all the research discussed in the present survey. Berry's background involves quantum chaology, a branch of physics which seeks to identify signatures of chaos in the spectra of physical systems on the border of the quantum and classical worlds. Random matrix theory and the Gaussian Unitary Ensemble in particular turn out to play a significant role in this. Having investigated the various connections between quantum chaos, random matrices and the nontrivial zeta zeros, Berry has conjectured that the nontrivial zeta zeros correspond to the spectrum of eigenvalues (energy levels) of a Hamiltonian governing a quantum mechanical system whose underlying classical mechanics are chaotic and time-irreversible. Remarkably, if such a dynamical system could be identified, that is, one whose spectrum corresponds exactly to the set of nontrivial zeta zeros, then the Hilbert–Pólya conjecture, and hence the Riemann hypothesis would be proven. An excellent, reasonably non-technical article documenting this quantum chaological approach to the zeta function, "A Prime Case of Chaos" by Barry Cipra, can be found on the American Mathematical Society website (in PDF format). More recently, a simpler popular exposition of these matters appeared in New Scientist (11/11/00) ([K]). The proof of the Riemann hypothesis has been called "the single most desirable achievement for a mathematician" ([G]), even compared to the holy grail. Astonishingly, one promising route to the problem is now a quest for a dynamical system which is in some sense "implied" by the distribution of prime numbers (via the intimately connected zeros of the zeta function). Berry refers to this hypothetical system as the Riemann dynamics. Needless to say, it has not yet been found, although many of its properties are known, if indeed it exists. The Berry–Keating H = xp heuristic has had a second life since. Sierra and Rodríguez-Laguna revisited the model concretely, obtaining a semiclassical spectrum matching the smooth part of the counting function of the zeros [SRL], and Bender, Brody and Müller proposed a PT-symmetric Hamiltonian whose eigenvalues, they argued, would be the imaginary parts of the nontrivial zeros [BBM]. That last claim attracted immediate and substantial criticism, and is not regarded as settled; it is included here because the argument it provoked is itself instructive. Connes, meanwhile, has continued along quite different lines, via a trace formula in noncommutative geometry [Cn]. In [K], Berry is quoted as says that if the dynamical system can be identified, then he is ". . . absolutely sure that . . . someone will find a clever way to make it in the lab. Then you'll get the Riemann zeros out just by observing its spectrum." ""Finding this system could be the discovery of the century," [Berry] says. It would become a model system for describing chaotic systems in the same way that the simple harmonic oscillator is used as a model for all kinds of complicated oscillators. It could play a fundamental role in describing all kinds of chaos. The search for this model system could be the holy grail of chaos. Until [it is found] we cannot be sure of its properties, but Berry believes the system is likely to be rather simple, and expects it to lead to totally new physics. It is a tantalising thought." (from [B]) Several interesting attempts have been made to produce the required Hamiltonian. J.V. Armitage has published notes [A] involving diffusion processes, Brownian motion and the Fokker–Planck equation. Bhaduri, Khare, et. al. have linked the problem to the scattering of partial waves in the analysis of resonances, e.g. in pion-nucleon scattering ([BhK]). The related work of Christopher Deninger ([D1-2]) studies dynamical systems (flows) on foliated manifolds. Much like the Selberg–Weil coincidence mentioned above, he has identified a similarity between
Helsinki physicist M. Pitkänen has suggested an approach involving superconformal invariance ([P]), and in a related work, C. Castro has argued that supersymmetric QM should instead be used, in combination with Brownian motion and p-adic fractal strings, to reduce the Riemann hypothesis to an inverse scattering problem ([C]). Both of these last two proposals sit outside the mainstream literature on the problem; they are archived here rather than endorsed. More references and commentary on the "spectral interpretation" of the Riemann zeta function can be found in the relevant section of the number theory and physics archive. p-adic numbers and fractal stringsAnother area of interconnection between number theory and physics, although of a very different flavour, involves the theory of p-adic numbers. Both A. Khrennikov and B. Dragovich have published impressive numbers of articles providing p-adic interpretations of physical systems and phenomena. This appears to be a rapidly developing area of research. Also of possible interest here are Castro and Mahecha's preprints [CM] and [C] linking the Riemann hypothesis to p-adic fractal strings and fractal p-branes. This has been partly inspired by Lapidus and van Frankenhuysen's fascinating book [LvF] which relates the zeta zeros to fractal geometry and a theory of complex dimensions. Castro explains: "Supersymmetry, p-adic stochastic dynamics, Brownian motion, Fokker–Planck equation, Langevin equation, prime number random distribution, random matrices, p-adic fractal strings, the adelic condition, etc...are all deeply interconnected in this paper." As above: archived, not endorsed. The Lapidus–van Frankenhuysen work these preprints draw on is a separate matter – that is mainstream mathematics. M. Pitkanen has also produced some intriguing notes suggesting a p-adic physics-inspired interpretation of the nontrivial zeta zeros, leading to his aforementioned proof strategy for the Riemann hypothesis which involves superconformal invariance, and in particular the Virasoro generator ([P]). Interestingly (although without any p-adic content), A. Petermann's preprint [Pe] also involves the Virasoro generator in an attempt to elucidate the deep reasons for the logarithmic distribution of primes. This involves a breaking of symmetry, namely that of scale invariance, and relates to certain aspects of quantum chromodynamics. Fields medalist Alain Connes has also linked the distribution of primes to a spontaneous symmetry-breaking, albeit of a different kind, in the article [BC] mentioned above. His extremely deep work involves noncommutative geometry, and the theory of adeles (related to p-adic analysis) in an attempt to produce the necessary Hamiltonian which will satisfy the Hilbert–Pólya conjecture discussed above. fractality and 1/f noiseIndirectly, as a result of studying nonlinear dynamics Marek Wolf discovered two instances of apparent fractality within the distribution of prime numbers ([W2-3]). These discoveries were realised experimentally using powerful computers. Wolf's resulting interest in the distribution of the primes led him to report numerical evidence for 1/f noise when the primes are treated as a "signal" in the sense of information theory ([W4]). This is also a self-similar (scale invariant, or fractal) property of the distribution of primes. 1/f noise, also known as flicker noise or pink noise, is a property of the power-frequency spectrum (obtained through Fourier analysis) of the signal. It has been detected in many diverse physical systems including sunspots, quasars, hourglasses, rivers, electronic components, DNA sequences, written language, weather patterns and stock exchange indices. It has been argued that its presence suggests some kind of "cooperative" effect over a wide range of timescales. Bak, Tang, and Wiesenfeld have offered an explanation for the ubiquity of 1/f noise by developing a simple model of self-organised criticality which has 1/f noise as a "temporal fingerprint" and self-similarity as a "spatial fingerprint" ([BTW]). The implication is that the previously mentioned physical systems could all be examples of self-organised critical systems. Wolf, being aware of this work, ended his article "1/f noise in the distribution of prime numbers" ([W4]) with the astonishing question "Are the prime numbers in a self-organized critical state?" I have since been informed that Bak et al. made a fundamental error in their calculations, and consequently the results in [BTW] apply to the less significant phenomenon of 1/f2 noise [JCF]. The wider 1980s hope, that self-organised criticality would account for the ubiquity of 1/f noise across physical systems, has not held up as a general principle either. Still, the possibility that the primes might constitute something akin to a self-organised system is one which I personally find to be quite compelling. Michel Planat of the Laboratoire de Physique et Métrologie des Oscillateurs du CNRS in France has since brought to light more connections between 1/f noise and the distribution of primes, via the Riemann Hypothesis ([Pl]). Ramanujan–Fourier series and the Wiener–Khintchine formulaAnother relatively recent development linking prime numbers and physics is an article by H. Gopalkrishna Gadiyar and R. Padma called "Ramanujan–Fourier series, the Wiener–Khintchine formula and the distribution of prime pairs" [GGP1]. Its abstract explains: "The Wiener–Khintchine formula plays a central role in statistical mechanics. It is shown here that the problem of prime pairs is related to autocorrelation and hence to a Wiener–Khintchine formula. "Experimental" evidence is given for this." The authors conclude with the following observation: "It is a pleasant surprise that the Wiener–Khintchine formula which normally occurs in practical problems of Brownian motion, electrical engineering and other applied areas of technology and statistical physics has a role in the behaviour of prime numbers which are studied by pure mathematicians." This follows the authors' earlier publication "Renormalisation and the density of prime pairs" [GGP2] which uses techniques from quantum field theory to suggest a possible approach to proving a number theoretic conjecture of Hardy and Littlewood. That conjecture – the asymptotic density of prime pairs – remains open. What has changed dramatically since is the neighbouring question of how small prime gaps can be, following Zhang's proof that infinitely many pairs of primes differ by less than a fixed bound [Z] and Maynard's substantially simpler method [My]. Neither result touches the asymptotic the authors above were after, which is worth noting precisely because it shows how far apart "infinitely often" and "how often" remain. quotations"Despite the stunning advances linking Riemann's zeta function to 20th century physics, no one is predicting an imminent proof of the Riemann hypothesis. Odlyzko's numerical experiments and evidence amassed by physicists have convinced everyone that a spectral interpretation of the zeta zeros is the way to go, but number theorists say they are at least one "big idea" away from even the beginnings of a proof. Mathematicians aren't yet sure what to aim at, says [Princeton University mathematician Peter] Sarnak." B. Cipra, from "A Prime Case of Chaos" "...the Riemann Hypothesis will be settled without any fundamental changes in our mathematical thoughts, namely, all tools are ready to attack it but just a penetrating idea is missing." Y. Motohashi, quoted in [S], p.228 "...there have been very few attempts at proving the Riemann hypothesis, because, simply, no one has ever had any really good idea for how to go about it." Atle Selberg, quoted in B. Cipra, "A Prime Case of Chaos" "I still think that some major new idea is needed here" E. Bombieri, quoted in [K] "Sometimes I think that we essentially have a complete proof of the Riemann Hypothesis except for a gap. The problem is, the gap occurs right at the beginning, and so it's hard to fill that gap because you don't see what's on the other side of it." H. Montgomery, quoted in [S], p.227 conclusionAll of this work taken together suggests to me that some wholly new, physics-inspired understanding of the prime numbers and their distribution is required, and possibly imminent. This might then provide the "major new idea" required for the proof of the Riemann hypothesis, whose persistent improvability neatly encapsulates the continuing mysteriousness of the primes, despite an ever expanding wealth of sophisticated analytic number theory.some ancient, dubious, speculative thoughts in this direction
references
|