Tim Harford for Financial Times on big data and how the same problems for small data still apply:The multiple-comparisons problem arises when a researcher looks at many possible patterns. Consider a randomised trial in which vitamins are given to some primary schoolchildren and placebos are given to others. Do the vitamins work? That all depends on what we mean by […]
…there are vastly more possible comparisons than there are data points to compare. Without careful analysis, the ratio of genuine patterns to spurious patterns – of signal to noise – quickly tends to zero.Tim Harford in the Financial Times has a great article called Big data: are we making a big mistake?.Share this:EmailFacebookGoogleGoogle+Matt MullenwegLike this:Like Loading…
Related
Visit site:
Statistical Validity in Big […]
Now that the U.S. government shutdown is behind us it is instructive to review what we went without for those weeks. Recently writing for the Financial Times, Robin Harding reported that experts feared that the U.S. government would be “flying blind” without access to the big data collected by the government, and which was imperiled by the shutdown.We would have been […]
