This seems to be the end of the road for LLM's. There's only so much accuracy on a highly non-linear space you can get from a regression.
If the Pareto rule is any indicator, 80% of results come from 20% of causes. It seems that we have alot more to learn about intelligence.
I am reminded of a statement on truth from an ancient philosopher, this sentiment seems to be exactly the opposite of the LLM training paradigm
“The seeker after truth is not one who studies the writings of the ancients and, following his natural disposition, puts his trust in them, but rather the one who suspects his faith in them and questions what he gathers from them, the one who submits to argument and demonstration and not the sayings of human beings whose nature is fraught with all kinds of imperfection and deficiency. Thus the duty of the man who investigates the writings of scientists, if learning the truth is his goal, is to make himself an enemy of all that he reads, and, applying his mind to the core and margins of of its content, attack it from every side. he should also suspect himself as he performs his critical examination of it, so that he may avoid falling into either prejudice or leniency.” - ibn al-Haytham
As long as there is upside or performance gaps that can be articulated, there is still room for improvement. And based on my own experience as well as all the comments I see, there is tons of upside remaining.
I really don’t think we know enough about what „intelligence“ is or how LLMs actually work to confidently say that this is the end of the road for LLM.
If you know all this, can you explain how these models produce advanced mathematical proofs? (as recently done by OpenAI, for example)
I tried to generate the next word to the best of my ability, starting with a mathematical problem, but I did not create a valid proof. How do these LLMs work when they create math proofs to problems not yet solved?
You don’t have the computational ability to process as many calculations as a datacenter. You can hardly transpose a 5×5 matrix in your mind, so you won’t be able to do what datacenters do.
This is like saying we don‘t know how a car works because a car can beat the best human athletes in 100 meter dash.
We know plenty about human cognition, and we know everything about how LLMs work. True we don’t know anything about intelligence but that is because “intelligence” is it self a fraught and vague term, and we haven’t (and perhaps never will) settled on what it means exactly.
There is high rate of progress in specific domains, not high rate of progress in generalness. The models haven't gotten generally smarter, for things they didn't focus on the models are just as bad as a year ago.
It's also important not to put too much faith into ancient sayings and aphorisms.
As a civilization, we are currently brushing up against the physics of efficiency. In many areas we have achieved close to what is theoretically possible, based on physics.
Such was not the case for the majority of human existence.
The body of research a.k.a. "writings of the ancients" is now insurmountably higher than it would have been during the time of ibn al-Haytham, when any kind of writing at all was scarce and literacy was low.
>This seems to be the end of the road for LLM's. There's only so much accuracy on a highly non-linear space you can get from a regression.
It's just inadequate benchmarks. Anyone who has used Fable for anything particularly difficult will have seen that it's miles ahead of Opus 5.0, yet the majority of benchmarks are completely unable to capture this.
Yes, and good senior software engineer is ahead of fable, but benchmarks can't capture that either.
We already know from testing humans that test scores don't correlate that well with how effective a person is at work. Same applies here, we just aren't that great at making good tests.
I work in AI evaluation, lots of problems and leakage is an issue as is ecological validity, but they definitely do not explain the progress we see.
I think Epoch has the best analysis I’ve seen on evaluation trends; they use IRT to basically model a variety of benchmark difficulties, and then model a capability parameter for each model. This is as robust a sort of “meta-study” of evaluations as I’ve seen and the trend in capabilities show no sign of slowing down.
So I think people’s feelings clash with reality, and that’s because releases are more frequent and the jumps between releases are smaller, but the growth in capabilities _over time_ has not changed for the better or worse over a very very long period of time.
Even on very small tests a fraction of questions might have wrong answers in the key.
If models can't get more than 90% of the benchmark right I think it's a strong indication that they were not trained on the answers and that benchmark itself is messy enough that <10% desired answers might be wrong or misleading.
Jumping in here, frankly I hate the trend of calling every type of automation intelligence.
Most "AI" is really an optimization algorithm in software tools, same as its always been. This really isnt anything new, aside from adding a chatbot / MCP interface to the same tools.
Talk about moving the goalposts. Pray tell, exactly what must an LLM do before you're willing to consider it AI? Be specific, otherwise you're just woo-mongering.
I started thinking about this back after the Llama 4 release, and since then our team has put a lot of thought into designing evaluations that don't saturate, are resistant to contamination, and can scale. What has worked best for us is using multi-agent environments with open-ended cooperative or competitive goals. Mostly designed as multiplayer games. The results tend to align with our experience for coding better than any non-aggregator benchmark, and likely at lower cost to run.
If you're ranking Opus > Fable you're ranking "do [clearly defined thing with easy to grade endpoint]" too much. Real world doesn't value that nearly as much and it's why benchmarks are maxxed.
I think its been pretty clear that in abnsense of clear use cases that are monetizable many model providers have been benchmaxxing on abstract or low utility average user performance.
This results in a lot of "oh wow it can do math I dont care about" and "it can't code a lot, but not well" outcomes instead of the core needs:
1) Cheaper faster and real time
2) Long walk capable without losing attention while rescoring goals over updated enviroment
3) Specific domain knowledge that can be trained quickly into the model (how we do work in this specific case)
If the Pareto rule is any indicator, 80% of results come from 20% of causes. It seems that we have alot more to learn about intelligence.
I am reminded of a statement on truth from an ancient philosopher, this sentiment seems to be exactly the opposite of the LLM training paradigm
“The seeker after truth is not one who studies the writings of the ancients and, following his natural disposition, puts his trust in them, but rather the one who suspects his faith in them and questions what he gathers from them, the one who submits to argument and demonstration and not the sayings of human beings whose nature is fraught with all kinds of imperfection and deficiency. Thus the duty of the man who investigates the writings of scientists, if learning the truth is his goal, is to make himself an enemy of all that he reads, and, applying his mind to the core and margins of of its content, attack it from every side. he should also suspect himself as he performs his critical examination of it, so that he may avoid falling into either prejudice or leniency.” - ibn al-Haytham
We know exactly how attention layers work and how they produce the next word as well as draw them from larger feature spaces.
I tried to generate the next word to the best of my ability, starting with a mathematical problem, but I did not create a valid proof. How do these LLMs work when they create math proofs to problems not yet solved?
This is like saying we don‘t know how a car works because a car can beat the best human athletes in 100 meter dash.
This is an amazingly ignorant thing to say given the current pace of progress.
As a civilization, we are currently brushing up against the physics of efficiency. In many areas we have achieved close to what is theoretically possible, based on physics.
Such was not the case for the majority of human existence.
The body of research a.k.a. "writings of the ancients" is now insurmountably higher than it would have been during the time of ibn al-Haytham, when any kind of writing at all was scarce and literacy was low.
It's just inadequate benchmarks. Anyone who has used Fable for anything particularly difficult will have seen that it's miles ahead of Opus 5.0, yet the majority of benchmarks are completely unable to capture this.
What were seeing is all models failing to ace these tests.
"Benchmark Saturation" is term that promotes lowering the bar.
We already know from testing humans that test scores don't correlate that well with how effective a person is at work. Same applies here, we just aren't that great at making good tests.
Frontier labs have categorically different & better set ups for evaluation, they're fine. It's work but it's not a crisis.
I think Epoch has the best analysis I’ve seen on evaluation trends; they use IRT to basically model a variety of benchmark difficulties, and then model a capability parameter for each model. This is as robust a sort of “meta-study” of evaluations as I’ve seen and the trend in capabilities show no sign of slowing down.
So I think people’s feelings clash with reality, and that’s because releases are more frequent and the jumps between releases are smaller, but the growth in capabilities _over time_ has not changed for the better or worse over a very very long period of time.
This is not "Acing" a test, this is hitting a wall.
If models can't get more than 90% of the benchmark right I think it's a strong indication that they were not trained on the answers and that benchmark itself is messy enough that <10% desired answers might be wrong or misleading.
Most "AI" is really an optimization algorithm in software tools, same as its always been. This really isnt anything new, aside from adding a chatbot / MCP interface to the same tools.
Oh! I got my name on a paper! I don't think there is much reward for it these days.
Data at https://gertlabs.com/rankings
This results in a lot of "oh wow it can do math I dont care about" and "it can't code a lot, but not well" outcomes instead of the core needs:
1) Cheaper faster and real time 2) Long walk capable without losing attention while rescoring goals over updated enviroment 3) Specific domain knowledge that can be trained quickly into the model (how we do work in this specific case)
> We find that nearly half of the our bench- marks exhibit saturation