Big data, same statistical challenges – FlowingData

Tim Harford for Financial Times on big data and how the same problems for small data still apply:The multiple-comparisons problem arises when a researcher looks at many possible patterns. Consider a randomised trial in which vitamins are given to some primary schoolchildren and placebos are given to others. Do the vitamins work? That all depends on what we mean by […]

Statistical Validity in Big Data | Matt Mullenweg

…there are vastly more possible comparisons than there are data points to compare. Without careful analysis, the ratio of genuine patterns to spurious patterns – of signal to noise – quickly tends to zero.Tim Harford in the Financial Times has a great article called Big data: are we making a big mistake?.Share this:EmailFacebookGoogleGoogle+Matt MullenwegLike this:Like Loading…
Related

Visit site: 

Statistical Validity in Big […]