Last Updated: July 15, 2025
Error Types
| H₀ True | H₀ False | |
|---|---|---|
| Reject H₀ | Type I Error (α) | Correct (Power = 1−β) ✓ |
| Fail to Reject H₀ | Correct ✓ | Type II Error (β) |
Power Determinants
| Factor | Effect on Power |
|---|---|
| Sample Size (n) ↑ | Power ↑ — larger samples reduce standard error |
| Effect Size (d) ↑ | Power ↑ — bigger effects easier to detect |
| Significance (α) ↑ | Power ↑ — looser threshold, but more false positives |
| Variance (σ²) ↑ | Power ↓ — noisier data masks effects |
Cohen's d (Effect Size)
| d Value | Interpretation | Example |
|---|---|---|
| 0.2 | Small | Height difference between 15 and 16-year-old girls |
| 0.5 | Medium | Visible to naked eye — height difference 14 vs 18-year-old girls |
| 0.8 | Large | Grossly perceptible — height difference 13 vs 18-year-old girls |
Sample Size Formulas (Two-Sample t-test)
| Target | Formula |
|---|---|
| Per Group | n = 2(Z₁₋α/₂ + Z₁₋β)² / d² |
| 80% Power, α=0.05 | n ≈ 16/d² (rule of thumb) |
| d=0.5 (medium) | n ≈ 64 per group |
| d=0.2 (small) | n ≈ 400 per group |
Python: Power Analysis
from statsmodels.stats.power import TTestIndPowerTwo-sample t-test power
analysis = TTestIndPower()Create analysis object
n = analysis.solve_power(effect_size=0.5, power=0.8, alpha=0.05)Required sample size per group
power = analysis.solve_power(effect_size=0.5, nobs1=100, alpha=0.05)Power given sample size
Pro Tip: 80% power is the industry standard minimum. Below 80%, you risk running an experiment that cannot detect a real effect. Above 95%, you may be wasting resources on oversized samples.