Your browser is out of date

This browser version is too old for Quizara to display the page correctly. Update to the latest version and refresh to continue.

P-Hacking

Updated · 2027 Level I · Quantitative Methods · Module 7 · Lesson 10 · 4 min lesson

On this page

What this lesson covers

P-hacking means altering an analysis until a result appears statistically significant, increasing the risk of false positive findings.

Interpret a p-value under the null hypothesis and compare it with significance. Recognize selective reporting, data mining and altered performance metrics, then use transparent reporting and out-of-sample validation.

Key points

  • P-value: The p-value is the smallest significance level at which the null would be rejected.
  • Right-tailed test: For a right-tailed test, the p-value is the probability of a statistic at or above the observed one.
  • Two-tailed test: For a two-tailed test with a symmetric statistic such as t, take the probability in one tail beyond the observed value's absolute size, and double it.
  • P-hacking: P-hacking inflates the probability of Type I errors, the false positives, and it gives statistical support to results that are not true.
  • Out of sample: Validate extensively out of sample, on data the strategy was not built on.

For example, Strategy Q’s right-tailed hurdle test gives p ≈ 0.041. It rejects H₀ at 5% but not at 1%. Removing its three worst months changes p to about 0.007; reporting only that revised sample to claim 1% significance is the lesson’s example of p-hacking.

Common mistakes

  • Re-running a test in new versions and reporting only the one that looks significant; each extra version gives luck another chance to push the p-value below the line.

Try a practice question

Choose an answer, then check it to read the explanation for every option.

Practice question

After a test of a strategy gives a p-value of 0.09, an analyst re-tests several versions of the strategy's rebalancing rule and reports only the version with a p-value of 0.03. This practice is most accurately described as:

Continue studying in Quizara

In Quizara the lesson comes with more practice questions on the topic, explanations for every option and an AI tutor for your follow-up questions.

Written by the Quizara Editorial Team under our editorial standards.