We present an approach for estimating the fraction of text in a large corpus<br>which is likely to be substantially modified or produced by a large language<br>model (LLM). Our maximum likelihood model leverages expert-written and<br>AI-generated reference texts to accurately and efficiently examine real-world<br>LLM-use at the corpus level. We apply this approach to a case study of<br>scientific peer review in AI conferences that took place after the release of<br>ChatGPT: ICLR 2024, NeurIPS 2023, CoRL 2023 and EMNLP 2023. Our results suggest<br>that between 6.5% and 16.9% of text submitted as peer reviews to these<br>conferences could have been substantially modified by LLMs, i.e. beyond<br>spell-checking or minor writing updates. The circumstances in which generated<br>text occurs offer insight into user behavior: the estimated fraction of<br>LLM-generated text is higher in reviews which report lower confidence, were<br>submitted close to the deadline, and from reviewers who are less likely to<br>respond to author rebuttals. We also observe corpus-level trends in generated<br>text which may be too subtle to detect at the individual level, and discuss the<br>implications of such trends on peer review. We call for future<br>interdisciplinary work to examine how LLM use is changing our information and<br>knowledge practices.