One of the most consequential decisions in experimental design is determining your sample size before collecting data. Yet, researchers frequently rely on rules of thumb or arbitrary targets, only to end up underpowered. When an experiment is underpowered, true effects vanish into noise, resources are wasted, and non-significant results become uninterpretable.
What is Statistical Power?
Statistical power (typically denoted as 1 – β) represents the probability that a statistical test will detect an effect of a given magnitude if that effect truly exists in the population. In formal terms, it is the probability of correctly rejecting a false null hypothesis ((H_0)).
Most empirical research targets a power of at least 80% (0.80) or 90% (0.90), keeping Type II error risk ((β)) at 10% to 20% alongside the traditional 5% ((α = 0.05)) Type I error threshold.
The Four Interconnected Pillars of Power
Every power calculation hinges on four mathematical parameters. If you fix any three, the fourth is deterministically resolved:
- Significance Level (α): The acceptable probability of a false positive (Type I error). Typically set at 0.05.
- Statistical Power (1 – β): The likelihood of avoiding a false negative (Type II error). Typically 0.80 or 0.90.
- Effect Size: The standardized magnitude of the true difference or relationship (e.g., Cohen’s d, Pearson’s r, Odds Ratios).
- Sample Size ((N)): The total number of observations or participants required.
The Peril of Underpowered Studies
Conducting research with inadequate power does not simply risk a “null result”—it distorts published literature in subtle ways:
- Inflated Effect Sizes (Winner’s Curse): When power is low (e.g., 20–30%), only studies with extreme effect size estimates cross the (p < 0.05) significance threshold, systematically exaggerating true effect estimates.
- Uninterpretable Negative Findings: If your statistical power was 30% and your test yields (p = 0.18), you have not provided evidence of absence; your experiment simply lacked the resolution to observe the phenomenon.
How to Calculate Sample Size Step-by-Step
Follow this rigorous workflow for prospective power analysis:
Step 1: Choose Your Primary Endpoint
Identify the single primary outcome variable and the exact hypothesis test that will analyze it (e.g., two-sample independent t-test, ANOVA, logistic regression, or mixed-effects model).
Step 2: Define the Minimum Meaningful Effect Size
Rather than estimating what effect you hope to see, determine the Smallest Effect Size of Interest (SESOI). What is the minimal difference that carries clinical, scientific, or practical significance?
Step 3: Conduct Prospective Calculations
Use open-source tools such as G*Power, the pwr package in R, or Python’s statsmodels.stats.power to run a prospective calculation tailored to your design.
# Example R snippet for independent two-sample t-test
library(pwr)
pwr.t.test(d = 0.5, power = 0.80, sig.level = 0.05, type = "two.sample")
Frequently Asked Questions (FAQ)
Can I run a post-hoc power calculation on observed data?
No. Retrospective (“post-hoc”) power calculated using observed effect sizes is mathematically redundant with the observed p-value and offers no diagnostic value. Power calculations must be planned prospectively.
What if participant recruitment is strictly constrained?
Perform a sensitivity power analysis to determine the smallest effect size your fixed sample size can reliably detect with 80% power, and assess whether that threshold is practically plausible.