In a recent academic paper from Google DeepMind, researcher Tom Zahavy asserts that current large language models (LLMs) lack the capability to trigger a true scientific revolution. According to the essay titled 'LLMs can't jump', these tools still lack the core cognitive mechanisms required to create entirely new knowledge for humanity.
Background & Causes
The explosive growth of large language models in recent times has sparked high expectations that AI could automate scientific discoveries. However, according to the paper by Google DeepMind's Tom Zahavy, current technology shows that LLMs operate primarily by restructuring and predicting existing data. They are incapable of making cognitive 'jumps' to establish revolutionary scientific theories. The study highlights that this lack of deep cognitive mechanisms limits this type of AI to a supporting role rather than acting as an independent creative agent.
Technical & Technological Analysis
Technically, large language models operate mainly by optimizing next-token prediction based on vast amounts of training data. This mechanism makes them exceptionally good at summarizing information, writing source code, or solving problems with precedents. However, to spark a scientific revolution, a system must be able to construct hypothetical models of the world and test them in reality. This is precisely where 'world models' are expected to excel. Unlike LLMs, which only process text, world models aim to deeply understand physical laws, cause-and-effect relationships, and simulate scenarios that have never appeared in their training data.
Expert Opinions & Insights
In the research paper 'LLMs can't jump', scientist Tom Zahavy argues that expecting LLMs to automatically generate new theories is unrealistic. The expert points out that complex cognitive mechanisms—such as self-refutation, abstract reasoning beyond language, and real-world interaction—are the missing pieces in current transformer network architectures. Tech analysts agree that while models like GPT or Claude are highly useful for research assistance, we need a fundamental shift in AI architecture to witness historic discoveries driven by computers.
Impact & The Future
The debate surrounding the limitations of LLMs versus the potential of world models is reshaping the direction of top AI research labs, including Google DeepMind. For the tech community and readers in Vietnam, this perspective offers a realistic view, helping to temper overinflated expectations regarding the current wave of generative AI. Instead of focusing solely on fine-tuning language models, future research trends are likely to shift heavily toward developing world models capable of interacting with and deeply understanding the surrounding physical environment.