What Am I Doing Wrong? Common Pitfalls in Rejection Sampling
In Cross Validated Categories, rejection sampling (the Accept-Reject algorithm) is a frequent topic for those moving beyond simple built-in distributions. If your simulated random numbers don't match your target Probability Density Function (PDF), you are likely falling into one of four classic traps. In 2026, as we use more complex simulation models, mastering these basics is essential for valid inference.
1. The Proposal Distribution (Envelope) Problem
The most common mistake is choosing a proposal distribution $g(x)$ that does not completely "cover" your target PDF $f(x)$.
- The Requirement: Your proposal $g(x)$ must have "thicker tails" than your target $f(x)$. If $f(x) > 0$ anywhere that $g(x) = 0$, you will never sample from that region.
- The Symptom: Your histogram looks correct in the center but is missing data at the edges (truncated tails).
2. Incorrect Scaling Constant ($M$)
Rejection sampling requires a constant $M$ such that $f(x) \leq M \cdot g(x)$ for all $x$.
- The Error: If $M$ is too small, the "envelope" $M \cdot g(x)$ will dip below the target PDF $f(x)$ at certain points. Any samples generated where $f(x) > M \cdot g(x)$ will result in an invalid distribution.
- Efficiency: If $M$ is too large, your algorithm will be valid but extremely slow, rejecting nearly every sample.
3. The Uniform Comparison Mistake
A frequent coding error occurs in the "Accept" step. The standard algorithm is:
Sample $x \sim g(x)$ and $u \sim \text{Uniform}(0, 1)$.
Accept $x$ if $u \leq \frac{f(x)}{M \cdot g(x)}$.
What goes wrong: Many beginners accidentally compare $u$ to $f(x)$ directly, or forget to divide by $M \cdot g(x)$. This destroys the probability logic and leads to a sample that follows the shape of $g(x)$ rather than $f(x)$.
4. Domain and Support Mismatches
In 2026, we often work with truncated distributions. If your target PDF is defined on $[0, \infty)$ but your proposal is a Normal distribution on $(-\infty, \infty)$, you must handle the negative values correctly. Simply "ignoring" them can bias your results if the scaling constant isn't recalculated for the truncated proposal.
| Mistake | Resulting Histogram | 2026 Fix |
|---|---|---|
| Low M value | Flat-topped or clipped peaks | Find $max(f(x)/g(x))$ using optimization. |
| Thin Tails | Missing outliers | Use a Cauchy or T-distribution as a proposal. |
| Ratio Error | Wrong shape entirely | Double-check the acceptance fraction math. |
5. Checking Your Work in 2026
On Cross Validated, the gold standard for verifying your simulation is the Kolmogorov-Smirnov (K-S) Test. This test compares your simulated sample against the theoretical Cumulative Distribution Function (CDF). If your p-value is very low, your rejection sampling logic is fundamentally flawed.
Conclusion
Rejection sampling fails when the math of the "envelope" doesn't respect the geometry of the target PDF. By ensuring $M \cdot g(x)$ always dominates $f(x)$ and verifying your acceptance ratio, you can simulate any complex distribution. For 2026 projects involving Monte Carlo methods, remember: an inefficient $M$ is better than an incorrect $M$. If you are still struggling, consider Adaptive Rejection Sampling, which builds the envelope dynamically as it learns the shape of the PDF.
Keywords
rejection sampling common errors 2026, accept-reject algorithm troubleshooting, proposal distribution requirements, scaling constant M in simulation, random number generation from PDF, Kolmogorov-Smirnov test for simulation, Cross Validated rejection sampling guide, 2026 Monte Carlo simulation tips.
