openverification.html

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>AI-accelerated Science Needs Open-sourced Verification — OpenAIReview</title>
<meta name="description" content="AI now produces mathematics faster than anyone can check it. We are launching OpenVerification, an open effort to find and confirm the errors in AI-generated papers, starting with the OpenAI math collection.">
<meta property="og:title" content="AI-accelerated Science Needs Open-sourced Verification">
<meta property="og:description" content="AI now produces mathematics faster than anyone can check it. We are launching OpenVerification, an open effort to find and confirm the errors in AI-generated papers, starting with the OpenAI math collection.">
<meta property="og:url" content="https://openaireview.org/openverification.html">
<meta property="og:type" content="article">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="AI-accelerated Science Needs Open-sourced Verification">
<meta name="twitter:description" content="AI now produces mathematics faster than anyone can check it. We are launching OpenVerification, an open effort to find and confirm the errors in AI-generated papers, starting with the OpenAI math collection.">
<!-- Google tag (gtag.js) -->
<script async src="https://www.googletagmanager.com/gtag/js?id=G-P2VZL2X4NX"></script>
<script>
  window.dataLayer = window.dataLayer || [];
  function gtag(){dataLayer.push(arguments);}
  gtag('js', new Date());
  gtag('config', 'G-P2VZL2X4NX');
</script>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700;800&family=Lora:ital,wght@0,600;0,700;1,600&display=swap" rel="stylesheet">
<style>
:root {
  --bg: #fafafa;
  --surface: #ffffff;
  --border: #e5e7eb;
  --text: #1a1a2e;
  --text-secondary: #6b7280;
  --text-muted: #9ca3af;
  --accent: #800000;
  --accent-light: #fdf2f2;
  --accent-hover: #600000;
  --radius: 12px;
  --radius-sm: 8px;
  --shadow-sm: 0 1px 2px rgba(0,0,0,0.04);
  --shadow-md: 0 4px 6px -1px rgba(0,0,0,0.07), 0 2px 4px -2px rgba(0,0,0,0.05);
}

* { box-sizing: border-box; margin: 0; padding: 0; }

body {
  font-family: 'Inter', -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
  background: var(--bg);
  color: var(--text);
  -webkit-font-smoothing: antialiased;
}

a { color: var(--accent); text-decoration: none; }
a:hover { color: var(--accent-hover); }

/* Header */
.header {
  background: var(--surface);
  border-bottom: 1px solid var(--border);
  padding: 0 32px;
  height: 64px;
  display: flex;
  align-items: center;
  gap: 24px;
  position: sticky;
  top: 0;
  z-index: 100;
}
.header .logo {
  font-size: 16px;
  font-weight: 700;
  color: var(--accent);
  letter-spacing: -0.3px;
}
.header-links {
  margin-left: auto;
  display: flex;
  gap: 20px;
  font-size: 14px;
  font-weight: 500;
}
.header-links a { color: var(--text-secondary); transition: color 0.15s; }
.header-links a:hover { color: var(--text); }
.header-links a { white-space: nowrap; }
@media (max-width: 480px) { div.header-links { gap: 12px; font-size: 12px; } div.header-links a.home-link { display: none; } }
.header-links a.current { color: var(--text); font-weight: 600; }

/* Blog layout */
.blog-hero {
  background: var(--surface);
  border-bottom: 1px solid var(--border);
  padding: 56px 32px 48px;
  text-align: center;
}
.blog-hero h1 {
  font-family: 'Lora', Georgia, serif;
  font-size: 34px;
  font-weight: 700;
  letter-spacing: -0.3px;
  line-height: 1.25;
  max-width: 680px;
  margin: 0 auto 20px;
}
.blog-hero .byline {
  font-size: 14px;
  color: var(--text-muted);
}

.blog-body {
  max-width: 720px;
  margin: 0 auto;
  padding: 48px 32px 80px;
}

/* Prose */
.blog-body p {
  font-size: 16px;
  line-height: 1.75;
  color: var(--text);
  margin-bottom: 20px;
}
.blog-body h2 {
  font-family: 'Lora', Georgia, serif;
  font-size: 24px;
  font-weight: 700;
  letter-spacing: -0.2px;
  margin: 48px 0 16px;
  color: var(--text);
}
.blog-body strong { font-weight: 600; }
.blog-body ul, .blog-body ol {
  padding-left: 24px;
  margin-bottom: 20px;
}
.blog-body li {
  font-size: 16px;
  line-height: 1.75;
  margin-bottom: 8px;
}
.blog-body img {
  width: 100%;
  border-radius: var(--radius);
  border: 1px solid var(--border);
  margin: 24px 0;
}

/* Table */
.blog-body table {
  width: 100%;
  border-collapse: collapse;
  font-size: 14px;
  margin: 24px 0;
}
.blog-body th {
  text-align: left;
  padding: 10px 14px;
  border-bottom: 2px solid var(--border);
  font-weight: 600;
  color: var(--text-secondary);
  font-size: 12px;
  text-transform: uppercase;
  letter-spacing: 0.4px;
}
.blog-body td {
  padding: 10px 14px;
  border-bottom: 1px solid var(--border);
}
.blog-body tr:last-child td { border-bottom: none; }
.blog-body tr:hover td { background: var(--accent-light); }

/* Example cards (shared with index.html) */
.examples-grid {
  display: grid;
  grid-template-columns: repeat(2, 1fr);
  gap: 16px;
  margin: 24px 0;
}
.example-card {
  background: var(--surface);
  border: 1px solid var(--border);
  border-radius: var(--radius);
  padding: 24px;
  transition: box-shadow 0.2s, border-color 0.2s, transform 0.15s;
  box-shadow: var(--shadow-sm);
  display: flex;
  flex-direction: column;
  text-decoration: none;
  color: inherit;
}
.example-card:hover {
  box-shadow: var(--shadow-md), inset 3px 0 0 var(--accent);
  border-color: #d1d5db;
  transform: translateY(-1px);
  color: inherit;
}
.example-card .card-field {
  font-size: 11px;
  font-weight: 600;
  text-transform: uppercase;
  letter-spacing: 0.5px;
  color: var(--accent);
  margin-bottom: 8px;
}
.example-card .card-title {
  font-size: 15px;
  font-weight: 600;
  line-height: 1.45;
  margin-bottom: 16px;
  flex: 1;
}
.example-card .card-meta {
  font-size: 12px;
  color: var(--text-secondary);
}
.count-row {
  display: flex;
  align-items: center;
  gap: 6px;
  margin-bottom: 14px;
  font-size: 11.5px;
  font-weight: 500;
}
.count-row .cr-label { color: var(--text-secondary); }
.count-row .cr-num { font-weight: 700; font-variant-numeric: tabular-nums; }
.count-row .cr-num.refine-num { color: #64748b; }
.count-row .cr-num.ai-num { color: var(--accent); }
.count-row .cr-sep { color: var(--text-muted); font-size: 14px; margin: 0 2px; }

/* Footer */
.footer {
  border-top: 1px solid var(--border);
  padding: 32px;
  text-align: center;
  font-size: 13px;
  color: var(--text-secondary);
}

@media (max-width: 768px) {
  .blog-hero h1 { font-size: 26px; }
  .examples-grid { grid-template-columns: 1fr; }
  .blog-body { padding: 32px 20px 60px; }
  .header { padding: 0 16px; }
}
</style>
</head>
<body>

<div class="header">
  <a class="logo" href="index.html">OpenAIReview</a>
  <div class="header-links">
    <a href="index.html" class="home-link">OpenVerification</a>
    <a href="reviewer.html">Review your paper</a>
    <a href="blogs.html" class="current">Blogs</a>
    <a href="https://github.com/ChicagoHAI/OpenAIReview" target="_blank">GitHub</a>
  </div>
</div>

<div class="blog-hero">
  <p class="byline" style="margin-bottom:18px;"><a href="blogs.html">&larr; All blogs</a></p>
  <h1>AI-accelerated Science Needs Open-sourced Verification</h1>
  <p class="byline">October 2026 &nbsp;&middot;&nbsp; Dang Nguyen</p>
</div>

<div class="blog-body">

<div style="background:var(--accent-light);border-left:3px solid var(--accent);border-radius:var(--radius-sm);padding:16px 20px;margin-bottom:28px;">
  <span style="font-size:11px;font-weight:700;text-transform:uppercase;letter-spacing:0.6px;color:var(--accent);">TL;DR</span>
  <p style="margin:6px 0 0;font-size:15px;line-height:1.6;">AI now produces scientific results faster than researchers can verify their correctness. We are launching <a href="index.html">OpenVerification</a>, a public platform that uses our open review tool to flag potential issues in AI-generated papers and invites researchers to verify them, starting with the OpenAI math papers.</p>
</div>

<p>In early October 2026, OpenAI released <a href="https://github.com/openai/math">a collection of 722 mathematical manuscripts</a> written by an unreleased internal model. The manuscripts are grouped into 372 families of results, and many of them claim solutions to well-known open problems. According to the release notes, the model was posed about 4,000 problems, and each result used about three hours of compute on average.</p>

<p>A month earlier, OpenAI announced a proof for the Navier-Stokes problem, with a Lean certificate, showing that a smooth fluid at rest can develop a singularity in finite time under the three-dimensional Navier&ndash;Stokes equations with a smooth external force.</p>

<p>These releases show the potential of AI to accelerate science. But a result only counts if we can be certain of its correctness, Without careful validation, a large volume of research output will <a href="blog.html">degrade the quality of science</a>.</p>

<h2>The first cracks are visible</h2>

<p><strong>The collection has already been corrected.</strong> One day after the release, OpenAI <a href="https://github.com/openai/math/blob/main/history.md">withdrew three manuscripts</a>. A sign error in one paper on Weil classes invalidated a cancellation argument. Two other papers on the Hodge conjecture depended on its construction. OpenAI revised 14 other manuscripts with proof repairs and corrected statements. The collection now has 719 manuscripts.</p>

<p><strong>Formal proofs do not settle the question either.</strong> About 42% of the main results in the collection come with a Lean formalization. For the Navier&ndash;Stokes result, <a href="https://arxiv.org/abs/2610.08144">Bastounis, Circelli, and Hansen</a> found that the Lean proof does not correspond to the proof written in natural language. A Lean certificate therefore does not guarantee a solution's correctness.</p>


<h2>Verification is now the bottleneck</h2>

<p>For most of the history of science, producing a result was more difficult and time-consuming than checking its correctness. But this is no longer the case. A model can produce a proof in a few hours, yet specialists may need weeks to check it, and for many of these papers only a few specialists in the world can.</p>

<p>The volume makes the problem worse. With more than 700 difficult manuscripts across 17 areas of mathematics, checking them needs experts from every one of those areas, and a journal often takes months to referee a single paper. Until that work is done, other researchers cannot safely build on these results, and much of the benefit of faster science is lost.</p>

<!-- <p>We believe that the verification of this body of work should be a collective effort with communication being as transparent and fast as possible.</p> -->

<h2>What we are launching</h2>

<p>To address this problem, we are launching <a href="index.html">OpenVerification</a>, a platform to identify errors in important AI-generated papers.</p>

<p>We use <a href="https://github.com/ChicagoHAI/OpenAIReview">OpenAIReview</a>, our open-source review tool, to read each paper and report potential issues. Every issue is shown on <a href="index.html">our website</a> next to the paper and linked to its location in the PDF.</p>

<p>So far we have reviewed 25 papers. The tool reported 121 potential issues in 22 of them, and labeled 19 issues as major. Most of them are steps that rest on a cited result or on another paper in the collection that the paper does not prove. A few claim that a step is wrong. We will continue to review more papers, and we invite the community to <a href="contribute.html">contribute reviews</a>.</p>

<!-- <p>An AI review system can be wrong, and the issues have not been confirmed by human mathematicians. This is where we need help.</p> -->

<h2>How you can help</h2>

<p><strong>If you are a mathematician, help us verify the comments.</strong> Open a paper in your area, read the issues, and mark each one as correct or incorrect. You can add a short note to let us know more detail. We will highlight the most important errors that the community confirms.</p>

<p><strong>Contribute a review.</strong> Many papers in the collection have no review yet. Our <a href="contribute.html">contribution guide</a> gives you a skill file to copy into a coding agent such as Claude Code or Codex. The agent reviews a paper of your choice, and you upload the result to our website.</p>

<p><strong>Help us improve the reviewer.</strong> OpenAIReview was built for papers in general, and mathematics has its own needs. If you see a kind of error that the tool misses, or a kind of comment that it should stop making, please contribute to the <a href="https://github.com/ChicagoHAI/OpenAIReview">GitHub repository</a>.</p>

<p>To ask questions and to talk with other contributors, join our <a href="https://discord.gg/5sPZbzYru">Discord server</a>.</p>

<h2>What comes next</h2>

<p>The OpenAI math collection is one of the first cases of science accelerated by AI, and more will follow, in mathematics and in other fields. Each one will raise the same question: which of these results can we build on?</p>

<p>Unchecked work cannot be a solid foundation on which to build our science, and a wrong result can cost years of work. Careful verification is what turns a fast stream of claims into reliable knowledge.</p>

<p>We start with these papers and the tool we have. Both will improve with every judgment and every contribution from the community. Let us verify the future of science together!</p>

<h2>Cite this post</h2>
<pre style="background:#f4f4f5;border:1px solid var(--border);border-radius:var(--radius-sm);padding:20px 24px;font-size:13px;line-height:1.7;overflow-x:auto;color:#374151;font-family:'Courier New',Courier,monospace;">@misc{openverification2026,
  title     = {AI-accelerated Science Needs Open-sourced Verification},
  author    = {Nguyen, Dang},
  year      = {2026},
  month     = {October},
  howpublished = {\url{https://openaireview.org/openverification.html}},
  note      = {OpenAIReview Blog}
}</pre>

</div><!-- .blog-body -->

<div class="footer">
  <span>&copy; 2026 OpenAIReview &nbsp;&middot;&nbsp; <a href="https://github.com/ChicagoHAI/OpenAIReview" target="_blank">GitHub</a></span>
</div>
</body>
</html>