Frontier reasoning models have exhibited incredible capabilities across a<br>wide array of disciplines, driven by posttraining large language models (LLMs)<br>with reinforcement learning (RL). However, despite the widespread success of<br>this paradigm, much of the literature has been devoted to disentangling truly<br>novel behaviors that emerge during RL but are not present in the base models.<br>In our work, we approach this question from a different angle, instead asking<br>whether comparable reasoning capabilites can be elicited from base models at<br>inference time by pure sampling, without any additional training. Inspired by<br>Markov chain Monte Carlo (MCMC) techniques for sampling from sharpened<br>distributions, we propose a simple iterative sampling algorithm leveraging the<br>base models' own likelihoods. Over different base models, we show that our<br>algorithm offers substantial boosts in reasoning that nearly match and even<br>outperform those from RL on a wide variety of single-shot tasks, including<br>MATH500, HumanEval, and GPQA. Moreover, our sampler avoids the collapse in<br>diversity over multiple samples that is characteristic of RL-posttraining.<br>Crucially, our method does not require training, curated datasets, or a<br>verifier, suggesting broad applicability beyond easily verifiable domains.