Nate Silver: “Buying Big Data to Solve Problems Is Oversold” | CFA …
At the 67th CFA Institute Annual Conference in Seattle, Nate Silver, founder of FiveThirtyEight, delivered an important message to financial professionals about the appropriate uses and definite limitations of big data that was well worth its weight in gold.
You would have to have been asleep for the last half-decade to not know that the business world, and consequently its investors, believe that big data is the next technological revolution to follow the internet. Yet Nate Silver delivered a stern warning to investors, “Buying big data to solve problems is oversold.”
Silver went on to say that with big data, the complexity of relationships increases at an exponential rate. He believes the real strength of big data is identifying relevant relationships. Then you can run a hypothesis on the data that you can test.
In other words: Understanding a more limited information set trumps misunderstanding a gigantic information set. To drive home his point, Silver stated, “If you are betting on a correlation, and you don’t understand causality, then I will take the other side of that bet, almost sight unseen.” It’s a point that all analysts looking for signal among noise should remember.
As an extreme example of overconfidence in the miracle of big data, Silver highlighted an article written in 2008 by Wired’s Chris Anderson, in which Anderson said that the ability to digest massive data obviated the need for scientific method. The idea being that with massive data, there is such transparency into the workings of the world that we no longer need the short cuts provided by scientific theory. Silver cautioned that this is overly enthusiastic and unrealistic and stated that as we get more data, we seem to get greater disagreement in model outputs. Wisely, he pointed out an example to investors that a trade in financial markets takes place because there is a disagreement in the market and not consensus.
Given that Silver came to international prominence for his ability to digest big data to accurately predict election results in the United States, his warning against abandoning scientific method and theories is an important one. To make Silver’s case that “big data” does not mean “little science,” he offered up the mortgage crisis of 2008–2009 as an example and pointed out that there were many flawed assumptions about mortgages and mortgage markets. Among the flaws were those made for default rates and correlations across different regions of the United States. In other words, big data and its crunching did not necessarily result in “big understanding.”
A further example offered by Silver was that of Japanese architectural engineering, which relied on a study stating that a 9.0 Richter scale earthquake was impossible. Consequently, buildings throughout Japan were built to only withstand a 8.6 magnitude earthquake. In other words, do not remove human judgment from the assessment of data; anticipate events that are unique and unusual that recent data may not incorporate. In fact, Silver stated, “It is amazing how often the real world delivers unique events.”
He went on to describe the remarkable results that even simple models can deliver — a seeming admonishment to the overcomplication of econometric models. He said that his election prediction models, the most accurate in history, were remarkably simple.
Silver started by taking the average of all of the polls in a region. He then accounted for a noise factor. Silver also knew that there were only 538 electoral votes available to candidates (and hence his firm’s namesake). He also knew from carefully assessing and judging state polling data that 11 of the 50 US states were 75× more important than the other 39 states in determining the outcome of the election. This was because of the high number of electoral college votes available, as well as the closeness of races in certain states. Though tempting, he resisted the urge to put undue emphasis on outlier polls that too strongly favored one candidate over another. Silver was then able to focus his attention on these critical states in forming his opinion.
From this simple model, Silver was able to make stunningly accurate predictions that flew in the face of national and international election coverage. Silver also pointed out to investors the important role that incentives have in shaping data outputs. For example, news broadcasters are in the business of keeping viewers tuned in, so broadcasters give undue attention to outlier polls in order to create viewer stickiness. He said the most accurate daily headline in elections is, “Things are more or less the same as yesterday; go watch sports instead.”
In summary, Nate Silver urged financial professionals not to hypnotize themselves with the shiny object of big data. He also urged analysts not to abandon scientific method and human judgment. Wise words from an analyst named Silver, spinning big data noise into signal gold.
Please note that the content of this site should not be construed as investment advice, nor do the opinions expressed necessarily reflect the views of CFA Institute.
Photo credit: W. Scott Mitchell
From:
Nate Silver: “Buying Big Data to Solve Problems Is Oversold” | CFA …
