In The Signal and the Noise Nate Silver sets himself a high aim, namely to tackle the tricky topic of prediction. To put it clearly, Silver attempts to explain how one can predict the future. No more no less. It might be tempting to conclude that this is a fairly high ambition, but Silver has some merit to his claims: Among others, he was one of the few to correctly predict the outcome of every single US state in the election in 2012, and for some time he even made his money on being able to correctly predict the outcome of baseball games in the US. He also correctly predicted a probable Biden in this years’ presidential election. This book, having become an instant classic in both the offices of executives in private companies as well as in the corridors of think tanks and intelligence agencies, is Silvers attempts to guide into the fascinating but complex world of how to be able to properly tell the future. In this, he leans heavily on his successful track record but is also using numerous examples to highlight some of the pitfalls one can fall into when attempting to predict the future. He also offers some advice on how to be able to improve one’s own predictions, among others by properly being able to use both quantitative as well as qualitative methods. The book, which originally was published in 2012, but will come out this year with a new preface.
«Experts» are often wrong
Silver has some compelling analyses concerning why predictions fail. As an example, he thoroughly examines the long-running US political talk show "The Mc Laughlin Group", and shows through analysis of over 1000 predictions given by the participating high ranking political experts that none of have been an more successful than simply rolling a dice: The outcome was around 50%, „ (…) meaning that they were about as likely to get a prediction right or wrong. They displayed as much political acumen as a barbershop quartet“. (Silver 2012, p.50).
Silver then explains why these (and several other) expert opinions fail, a fact he attributes mostly to human factors: Bipartisanship, matters of wanting to believe in certain narratives and as the title states, that the important news drown in the noise of too much information.
There can be such a thing as too much data
And concerning the topic of the enormous amount of information available in todays society Silver shares a point which some might find rather surprising: Rather than embracing big data and algorithms to be the „cure-it-all“ for navigating through the murky waters of data overload, he focusses on its limits. By drawing on examples from the financial crisis in 2008, the terror attacks on the US in 2001 and the fall of the Soviet Union, he shows how neither of these were properly predicted, although there was data to support it and substantial quantitative processing power to go around. Already in the introduction Silver states that "The numbers have no way of speaking for themselves. We speak for them" (Silver 2012, p.9).
The Signal and the Noise can with benefit be read alongside another Penguin-published book hit of the last decade, namely Daniel Kahneman’s Thinking, Fast and Slow. (2011) The main similarities between these two books is that they combine scientifically based methods to enlighten phenomena that have implications for most people in todays world. And the successes of both books lie in the fact that they do this in a way that is accessible for the interested layman, although it must be noted, a persistent one.
Earthquake and the flu
Silver goes on to cover a range of topics, and one of the most interesting points he makes is how different the predictive methods used are when attempting to say anything relevant about future developments in different fields. Some fields, such as weather reports, have the last forty years greatly increased their accuracy, whereas e.g. earthquake predictions seem to have gotten all but nowhere.Of particular interest in Covid 19-ridden times, should be Silver’s chapter on predictions of epidemics. Employing data both from the Spanish Flu in 1916-1918 as well as other flu epidemics, he clearly explains why flu predications always fail. The reasons for this are manifold, but of among the most intriguing is the phenomenon of self-cancelling predictions, which occur when a certain phenomena is being identified so early that successful countermeasures ensures it never comes to fruition. This point should be clearly recognisedacross a lot of sectors concerned with both safety and security in general as well. In addition, Silver sheds light on terms and topics which have been used rather widely in all media outlets these last months. The (in)famous R-number, which „measures the number of uninfected people that can expect to catch a disease from a single inflected individual“ (Silver 2012, p.214). Most countries base a substantial amount of their policies to counter the pandemic on this number. Silver, on the other hand, states that the reproduction "can usually not be formulated until well after a disease has swept through a community and there has been sufficient time to scrutinise the statistics."(Silver 20912, p.215). But the most important point follows: "So epidemiologists are forced to make extrapolations about it [i.e. a pandemic] from a few early data points."(ibid). If this point had been better understood when the pandemic broke out, many misunderstandings and also the distrust and downright contempt towards scientific methods shown by certain leading politicians in the Western world, might have been avoided. And a large number of lives might have been saved.
Extrapolation is assuming
A scientific point which Silver makes in his chapter devoted to epidemics which deserves its fair share of attention, is the limits of extrapolation in prediction. As should be generally known, extrapolation is the most used way of prediction, and is based on a belief that if all factors remain identical, any current trend will continue. Silver argues that extrapolation is usually used in a way that one assumes that all the trends will continue indefinitely, resulting in some of the best-known failures of prediction, among them in the number of AIDS and HIV-numbers, which were 100% higher than the predictions that were made in the 80s. However, the most important point in this regard is that "precise predictions aren’t really possible to begin with when you are extrapolating on an exponential scale."(Silver 2012, p. 213). Applied to the AIDS-example, a proper extrapolation in this case would result in numbers ranging from 35 000 to 1, 8 million cases due to the margin of error. As Silver dryly adds: "That’s much to broad a range to provide for much in the way of predictive insight" (ibid)
Can Thomas Bayes predict whether your partner is cheating on you?
Further topics in the book cover chaos theory and complexity theory, as well as Bayesian probability, the latter explained though a series of downright entertaining examples: "what is the probability that your partner is cheating on you?" (Silver 2012, p.243) or more sombre ones such as the probability of a terror attack using planes to crash into a skyscraper on Manhattan.(Silver 2012, p.247)
But this lightheartedness should not be taken as an example of this book being a sort of infotainment book, quite on the contrary, Although Silver has a dry, sardonic wit which permeates through the whole book, and while it in every way provides for a stimulating read, the theoretical concepts presented in this book are no light matter. It is possible to read this book superficially, but the full benefit of the concepts presented herein manifest themselves when every chapter is being thoroughly read and understood. This might pose a challenge for some in these internet-ridden times when the average concentration span has been somewhat deteriorating. In addition, some of the chapters might at first glance not be seen as relevant for European readers (statistics in baseball being the most obvious example). However, Silver’s book is built upon an introduction of different quantitative and qualitative in each chapter, and readers who put the extra effort in will gain great rewards by indulging also in topics they might not have been interested in beforehand - to put it this way, reading a regular dull weather report will never be dull again after this book.
