Top physicist says chatbots are just ‘glorified tape recorders’
Top physicist says chatbots are just ‘glorified tape recorders’
Top physicist says chatbots are just ‘glorified tape recorders’::Leading theoretical physicist Michio Kaku predicts quantum computers are far more important for solving mankind’s problems.
You're viewing a single thread.
I call them glorified spread sheets, but I see the correlation to recorders. LLMs, like most "AIs" before them, are just new ways to do line of best fit analysis.
That's fine. Glorify those spreadsheets. It's a pretty major thing to have cracked.
It is. The tokenization and intent processing are the thing that impress me most. I've been joking since the 90's that the most impressive technological innovation shown on Star Trek TNG was computers that understand the intent of instructions. Now we have that... mostly.
To counter the grandiose claims that present-day LLMs are almost AGI, people go too far in the opposite direction. Dismissing it as being only "line of best fit analysis" fails to recognize the power, significance, and difficulty of extracting meaningful insights and capabilities from data.
Aside from the fact that many modern theories in human cognitive science are actually deeply related to statistical analysis and machine learning (such as embodied cognition, Bayesian predictive coding, and connectionism), referring to it as a "line" of best fit is disingenuous because it downplays the important fact that the relationships found in these data are not lines, but rather highly non-linear high-dimensional manifolds. The development of techniques to efficiently discover these relationships in giant datasets is genuinely a HUGE achievement in humanity's mastery of the sciences, as they've allowed us to create programs for things it would be impossible to write out explicitly as a classical program. In particular, our current ability to create classifiers and generators for unstructured data like images would have been unimaginable a couple of decades ago, yet we've already begun to take it for granted.
So while it's important to temper expectations that we are a long way from ever seeing anything resembling AGI as it's typically conceived of, oversimplifying all neural models as being "just" line fitting blinds you to the true power and generality that such a framework of manifold learning through optimization represents - as it relates to information theory, energy and entropy in the brain, engineering applications, and the nature of knowledge itself.
Ok, it's a best fit line on an n-dimentional matrix querying a graphdb ;)
My only point is that this isn't AGI and too many people still fail to recognize that. Now people are becoming disillusioned with it because they're realizing it isn't actually creative. It's still still just a fancy comparison engine. That's not not world changing, but it's also not Data from Star Trek
I get that, but what I'm saying is that calling deep learning "just fancy comparison engine" frames the concept in an unnecessarily pessimistic and sneery way. It's more illuminating to look at the considerable mileage that "just pattern matching" yields, not only for the practical engineering applications, but for the cognitive scientist and theoretician.
Furthermore, what constitutes being "actually creative"? Consider DeepMind's AlphaGo Zero model:
Mok Jin-seok, who directs the South Korean national Go team, said the Go world has already been imitating the playing styles of previous versions of AlphaGo and creating new ideas from them, and he is hopeful that new ideas will come out from AlphaGo Zero. Mok also added that general trends in the Go world are now being influenced by AlphaGo's playing style.
Professional Go players and champions concede that the model developed novel styles and strategies that now influence how humans approach the game. If that can't be considered a true spark of creativity, what can?