Big data, same statistical challenges – FlowingData

Tim Harford for Financial Times on big data and how the same problems for small data still apply:The multiple-comparisons problem arises when a researcher looks at many possible patterns. Consider a randomised trial in which vitamins are given to some primary schoolchildren and placebos are given to others. Do the vitamins work? That all depends on what we mean by […]

Statistical Validity in Big Data | Matt Mullenweg

…there are vastly more possible comparisons than there are data points to compare. Without careful analysis, the ratio of genuine patterns to spurious patterns – of signal to noise – quickly tends to zero.Tim Harford in the Financial Times has a great article called Big data: are we making a big mistake?.Share this:EmailFacebookGoogleGoogle+Matt MullenwegLike this:Like Loading…

Visit site: 

Statistical Validity in Big […]

Soaring High Without Big Data – Ludwig von Mises Institute Canada

Now that the U.S. government shutdown is behind us it is instructive to review what we went without for those weeks. Recently writing for the Financial Times, Robin Harding reported that experts feared that the U.S. government would be “flying blind” without access to the big data collected by the government, and which was imperiled by the shutdown.We would have been […]