Statistical Test Chooser
Work out which statistical test fits your data from the question, the number of groups, the data type and whether the assumptions hold, with the non-parametric alternative for each.
Last reviewed by the Radiatus Cloud team
Need this done properly for your business?
Radiatus delivers secure cloud, DevOps & compliance engineering.
The test follows from the design, not the other way round
Choosing a test after collecting data is choosing among the options the design left available, and the design should have been chosen with the analysis in mind. Deciding the test first also prevents the most common form of analytical drift, where several tests are tried and the one giving the smallest p-value is reported. That practice inflates the error rate substantially and is invisible in the write-up unless it is disclosed.
Paired and unpaired is the distinction most often got wrong
Measurements are paired when each observation in one group is naturally matched with one in the other: the same subject before and after, the same specimen under two conditions, or matched pairs by design. Pairing removes the variation between subjects, which makes the test substantially more powerful. Analysing paired data as unpaired throws that away and usually fails to find a real effect; analysing unpaired data as paired is simply invalid.
Non-parametric tests are not free
They avoid the normality assumption and they test a different hypothesis, usually about ranks or medians rather than means, and they have less power when the parametric assumptions do hold. Reaching for one automatically because the data "might not be normal" costs real ability to detect an effect. With reasonably large samples the parametric tests are robust to moderate non-normality, which is often forgotten.
Related tools
- Security Term Translator — Translate complex security jargon into business language.
- Security Myth vs Reality — Fact-check common security myths.
- Breach News Impact — Contextualize major breaches into understandable analogies.
- Config Safety Analyzer — Paste a config snippet to check for obvious safety issues.
Frequently Asked Questions
Should I test for normality first?
Usually not, and it is common practice. Formal normality tests have almost no power on small samples and reject trivial deviations on large ones, which is the opposite of what is useful. Looking at a plot and considering the measurement usually decides it better.
When is data paired?
When each observation in one group is naturally matched with one in the other: the same subject twice, the same specimen under two conditions, or matched by design. Pairing removes between-subject variation and makes the test considerably more powerful.
Why not always use a non-parametric test?
Because they test a different hypothesis, usually about ranks rather than means, and they have less power when the parametric assumptions hold. Reaching for one automatically costs real ability to detect a genuine effect.
What if I have more than two groups?
Use an analysis of variance rather than several two-group tests. Running every pairwise comparison inflates the chance of a false positive: with five groups there are ten comparisons, and at the usual threshold that is a 40 percent chance of at least one by luck.
Does choosing the test after seeing the data matter?
Considerably. Trying several tests and reporting the one with the smallest p-value inflates the error rate well beyond the stated level, and it is invisible in a write-up unless disclosed. Decide the test from the design before collecting.
Privacy & Security
Everything runs in your browser; nothing is uploaded.
How to Use
Answer the questions about your data to see which test applies.
Disclaimer: This tool is provided "as is" without warranty of any kind. Results are for educational and utility purposes.