OpenAI’s internal model Astra solved ten long-standing open problems in mathematics and theoretical computer science. Noam Brown called it “a major step for scientific reasoning.” The total cost of the tokens used: about $2,000 at Sol API rates.
That is genuinely impressive. It is also, according to cognitive scientist Gary Marcus, being interpreted with a deeply flawed argument — a fallacy that keeps repeating every time an AI lab announces a breakthrough.
The reasoning error has a name: the fallacy of composition. It is the assumption that because a system is great at one kind of task, it is great at all tasks — that a model that shines at formal math is close to solving everything. Astra is very good at certain kinds of mathematics. That says almost nothing about whether it can reliably read a PDF, follow hard rules, or stop hallucinating.
The Fallacy at the Center of the Celebration
Within hours of OpenAI’s announcement, prominent voices were extrapolating wildly. One commentator wrote that “everyone in the world will soon be able to use the model that made these breakthroughs for every problem they face in life.” Another claimed “the species just crossed a one-way threshold.” Elon Musk took it as evidence we had reached the Singularity.
Marcus points out that all of these claims commit the same logical error. The fallacy of composition runs like this: assume all cognition is created equal, then whenever AI succeeds at one impressive form of cognition, conclude that success on all forms of AI is imminent.
This does not follow. Expertise in mathematics does not guarantee genius in writing, or understanding human relationships, or navigating open-ended real-world situations. That is precisely why psychologists like Howard Gardner and Robert Sternberg built multidimensional theories of intelligence, and why the SAT tests math separately from verbal reasoning. A person can be brilliant at algebra and struggle with a conversation.
There is no reason to think a model is different.
Why Math Is the Special Case
This is not just about logical caution. There is a principled reason to think math success will not generalize as much as people hope.
Math lends itself to two things that most real-world tasks do not: verification and massive cheap synthetic data.
- You can verify a math proof using symbolic tools. Every step can be checked mechanically.
- You can generate as many math problems as you want, with guaranteed-correct answers, at virtually no cost.
The same is true for coding. It is not true in general. You cannot simulate the open-ended world. You cannot verify a military strategy the way you verify a proof. You cannot generate infinite reliable training data for a customer-service conversation or a legal argument or a medical diagnosis.
Marcus has been making this point since at least January 2025, echoing what he and Ernie Davis argued about Go back in 2019 — that AlphaGo’s mastery of one board game would not be a panacea. The lesson is old: success in one domain does not guarantee success in all.
The most vivid historical example: IBM’s Watson crushed Jeopardy! in 2011, and then famously stumbled when IBM tried to turn it into a cancer-fighting medical assistant. Expertise at one task, no matter how impressive, does not transfer automatically.
What OpenAI Has Not Shown
There is a second, more uncomfortable point. OpenAI’s announcement — the tweet, the blog post, and the 249-page mathematics paper — reveals almost nothing about how Astra works.
Marcus’s critique is specific: the paper describes new math results, but not one page explains how the model works, how the proofs were verified, what role humans played, or whether any of the proposed proofs contained errors.
“Yesterday’s tweet and blog were marketing, not science,” he wrote.
This matters because extraordinary claims need extraordinary evidence. The math results are formalized in Lean, which means the proofs themselves are mechanically checkable — that part is real. But the gap between “this model produced verifiable math proofs” and “this model is on the path to general intelligence” is enormous, and OpenAI has not bridged it publicly.
The Two Things That Can Be True at Once
The honest position holds both halves together.
What is true: Astra solved ten genuinely hard problems. The proofs carry Lean certificates, which means a computer verified every step. That is a real milestone in scientific reasoning — the $2,000 cost per ten problems is striking, and the fact that a model can produce machine-checkable proofs at scale is meaningful.
What is also true: this does not make Astra AGI, ASI, or anywhere close. Marcus says he will be surprised if Astra scores even 5 out of 10 on his 2024 bet with Miles Brundage about AI progress. Success at formalizable math does not imply success at the messy, open-ended, unverifiable tasks that dominate most of human work.
Both of these are facts. The mistake is dropping either one.
The Verification Gap, Concretely
To make the argument less abstract, think about what “verifiable” actually means in practice.
When Astra produces a math proof, a tool like Lean checks every inference step. The model cannot quietly fudge a missing step, because the verifier will not accept it. That is a real constraint, and it is exactly why the ten results are credible.
Now consider a task where verification is not mechanical. Say you ask a model to draft a response to an upset customer, summarize a legal contract, or evaluate whether a startup idea is sound. There is no Lean equivalent for any of these. Nobody can write a tool that mechanically confirms “yes, this customer reply is correct” or “yes, this contract summary captured all material clauses.” The only check is a human reading it — which is slow, expensive, and still error-prone.
That is the structural difference. Math and code have machine-checkable ground truth. Most of the world does not. Astra’s success lives in the first category, and the people extrapolating it to the second are skipping over exactly the part that does not transfer.
What This Means for You
If you are a developer or a scientist: Astra’s math results are worth paying attention to as a genuine capability. Machine-verified proofs at this cost could accelerate parts of research. But plan around what it has proven — formalizable domains with cheap verification — not around unproven general reasoning.
If you follow AI news: be suspicious of “the Singularity is here” framing. The pattern is predictable: impressive result, maximal extrapolation, little scrutiny of the gap between them.
If you make decisions based on AI capabilities: the practical takeaway is calibration. A model that is great at math is not automatically great at everything else. Test it on your actual task before assuming it can do it — especially for anything involving reliability, rules, or open-ended judgment, where verification is hard.
A Brief Note on the Cost Claim
One number deserves a closer look, because it is doing a lot of work in the celebration: the “$2,000 cost.”
It is true that, at Sol API rates, the tokens used to generate all ten solutions would have cost about $2,000. That is a real and striking figure. But it is also a number that can mislead. The $2,000 reflects only the tokens consumed by the final successful runs. It does not include the model’s training cost, the compute used for earlier failed attempts, or the human work of turning raw model output into the polished proofs and papers. Research-and-development costs are not the same as inference costs.
None of this diminishes the achievement. But when a number becomes the centerpiece of a story about how cheap AGI has become, it is worth remembering what it does and does not measure. Inference cost is not total cost, and total cost is not capability.
Bottom Line
Astra is genuinely impressive, and the math results are real. But the leap from “great at formal mathematics” to “great at everything, and the species crossed a threshold” is a logical error — the same fallacy AI enthusiasts have repeated after every major advance for years.
The math results are verified. The extrapolations are not. Keeping those two facts separate is the entire game.