Pre-training as we know it will end

“We have reached the pinnacle of data and there will be no more information,” Ilya Sutskever said. “We have to manage the information we have because there is only one Internet.”
He predicts the next-generation model will “look truly agentic.” In addition to being “representative,” he also said that in the future, AI systems will be able to use reasoning. Unlike today’s AI, which mostly pattern-match from what models have seen before, AI systems in the future will be able to solve problems step-by-step in a manner comparable to real thinking.
“The more rational a system, the more unpredictable it is,” Sutskever said. He compared the unpredictability of “a truly rational system” with advanced chess AI, where “the best chess player is unpredictable.”
“They will understand things from a limited amount of information,” he said. “They will not be confused.”
On stage, he compared AI development with evolutionary biology, citing research showing the relationship between brain mass and body mass in each animal species. He noted that even though most mammals have similar scaling patterns, hominids (human ancestors) showed a distinctly different steepness in ratio of brain mass to body mass when measured on scallogarithm scale.
He suggests that, just as evolution discovered a new scaling model for the brain of hominids, AI may discover a new scaling method that goes beyond current pre-training models.
@NueroIPS2024 #AI #ilyasutskever
Source: The Verge
Link : https://www.facebook.com/share/p/14tAaMi5cq/