The Signal and the Noise

One of the differences between a textbook and a well-written non-fiction novel is their emphasis on applications. Textbooks are well-known for being very cut and dry, with a few generic “examples” for students to practice on. To get a real taste on where these theories are applied, one must venture elsewhere. Thus, if I was a professor, I would recommend a textbook and a novel for my students to read side-by-side.

If I taught statistics and forecasting, I would definitely put this on the top of my novel choices. I’m just going to put it right out there:

The Signal and the Noise is very well written.

img_2358

Nate Silver takes some of the most prevalent theories in probability and applies them to our everyday lives. Some of these topics the author has had first-hand experience in, including sports betting and political campaigns. Others topics not in his background are analyzed with similar theories but adapted to their own methodologies. What makes each of these topics different from one another, and how well we can predict them, is about the known data and theory that is applied to statistical models.

One of the most well-celebrated achievements in applied statistics is meteorology. Humankind has come a long way in modeling how the weather behaves based on “initial conditions” and how those will impact tomorrow’s forecast. In addition, we can easily collect essential input data that our model requires (including temperature, humidity, pressure). Thus, we can accurately predict  where and when hurricanes and storms will hit.

This is in contrast to stock market pricing, which the data and theories are very noisy and unknown. It’s one of the reasons why it’s best just to invest in the S&P 500 [what some may call “boring” stocks] than it is to pitch it towards “active investors;” no one can consistently beat the market without utilizing inside information [see my review on the novel Black Edge].  In addition, the stock market will react and change based on your predictions. This will change the price of the stock because you created demand. When one buys a stock, regardless of how good it is, it looks more favorable and the price typically rises. And when it rises, other people may buy that stock. This can result in a perpetuated cycle that can inflate the bubble and continue at the incentive of other’s financial’s interest until the system can’t ignore the truth….and the bubble bursts. Everyone rushes their money out, and the price plummets on a stock that no one wants to buy anymore.

And then there are topics where we are desperately seeking signal. Earthquakes and acts of war/terror are such examples and are covered in their own chapters. There’s no successful theory to understand when these occurrences will, but we can nevertheless theorize insight on how often such events will occur. We can look back and understand how often magnitude 8 earthquakes happen in each area of the US, and this information can give us financial insight on how well to reinforce new buildings in each city.

….. I could write a lot more on the other topics covered, but then there would be no incentive to read the book anymore.

The book does have a lot of insights, and it is never biased. For example, on the chapter on global warming, the author’s two cents on if it exists is ‘yes….and somewhat no.’ There are a lot of side views to each arguments, almost playing devil’s advocate on occasion to truly create a neutral viewpoint. In contrast, this does lengthen the chapters, and discussions can be drawn out at times. Around the last 20% of the book, I’ve found myself skimming a bit more than I felt proud of admitting to. But there are plenty of images and graphs to illustrate the points that Nate wants to relay to his audience. And there are a lot of graphics, roughly 1 image for every 3-5 pages, which is definitely a good thing and constantly steers you back into on topic.

img_2360-e1552265076138.jpg

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s