Frontier reasoning models have exhibited incredible capabilities across awide array of disciplines, driven by posttraining large language models (LLMs)with reinforcement learning (RL). However, despite the widespread success ofthis paradigm, much of the literature has been devoted to disentangling trulynovel behaviors that emerge during RL but are not present in the base models.In our work, we approach this question from a different angle, instead askingwhether comparable reasoning capabilites can be elicited from base models atinference time by pure sampling, without any additional training. Inspired byMarkov chain Monte Carlo (MCMC) techniques for sampling from sharpeneddistributions, we propose a simple iterative sampling algorithm leveraging thebase models' own likelihoods. Over different base models, we show that ouralgorithm offers substantial boosts in reasoning that nearly match and evenoutperform those from RL on a wide variety of single-shot tasks, includingMATH500, HumanEval, and GPQA. Moreover, our sampler avoids the collapse indiversity over multiple samples that is characteristic of RL-posttraining.Crucially, our method does not require training, curated datasets, or averifier, suggesting broad applicability beyond easily verifiable domains.