On this page
What this lesson covers
P-hacking means altering an analysis until a result appears statistically significant, increasing the risk of false positive findings.
Interpret a p-value under the null hypothesis and compare it with significance. Recognize selective reporting, data mining and altered performance metrics, then use transparent reporting and out-of-sample validation.
Key points
- P-value: The p-value is the smallest significance level at which the null would be rejected.
- Right-tailed test: For a right-tailed test, the p-value is the probability of a statistic at or above the observed one.
- Two-tailed test: For a two-tailed test with a symmetric statistic such as t, take the probability in one tail beyond the observed value's absolute size, and double it.
- P-hacking: P-hacking inflates the probability of Type I errors, the false positives, and it gives statistical support to results that are not true.
- Out of sample: Validate extensively out of sample, on data the strategy was not built on.
For example, Strategy Q’s right-tailed hurdle test gives p ≈ 0.041. It rejects H₀ at 5% but not at 1%. Removing its three worst months changes p to about 0.007; reporting only that revised sample to claim 1% significance is the lesson’s example of p-hacking.
Common mistakes
- Re-running a test in new versions and reporting only the one that looks significant; each extra version gives luck another chance to push the p-value below the line.
Try a practice question
Choose an answer, then check it to read the explanation for every option.
Continue studying in Quizara
In Quizara the lesson comes with more practice questions on the topic, explanations for every option and an AI tutor for your follow-up questions.
Written by the Quizara Editorial Team under our editorial standards.