As a full-time scientist you test a lot of hypothesis—a lot more than one per week. Every sample or condition you compare in an experiment is a hypothesis test. If you test tens or hundreds or even thousands of samples in a single experiment, it’s guaranteed that you’ll find many false positive results, in every experiment. So what can you do about it? You can calculate your False Discovery Rate (FDR), which tells you what fraction of your accepted tests are false.
FDR is a very simple concept. It is the number of false discoveries in an experiment divided by total number of discoveries in that experiment. But there is a problem, you never know how many discoveries are actually real or false when you accepted them. So how do you estimate FDR from your data?
Family-Wise Error Rate (FWER) is an approach for multiple testing correction. It is a scary sounding term, but don’t be deterred. It’s simply the probability that one or more of your “family” of multiple tests is false. In my view, FDR is generally more aligned with what you want as an experimenter. But FWER offers an alternative, generally more stringent, approach to reduce your false findings.
Since the time of Newton and Galileo, the tools for capturing and communicating science have remained conceptually unchanged. But these tools are wholly inadequate for the complexity of today’s scientific challenges. Riffyn created the Scientific Development Environment to change that.