“Model collapse” threatens to kill progress on generative AIs
“Model collapse” threatens to kill progress on generative AIs
Generative AIs start churning out nonsense when trained on synthetic data — a problem that could put a ceiling on their ability to improve.
have we tried feeding them actual human beings yet ?
Two outcasts among their peers, Gary Wallace and Wyatt Donnelly spent a good deal of their youth as pioneers and early adopters of AI.
Every single one of us, as kids, learned the concept of "garbage in, garbage out"; most likely in terms of diet and food intake.
And yet every AI cultist makes the shocked pikachu face when they figure out that trying to improve your LLM by feeding it on data generated by literally the inferior LLM you're trying to improve, is an exercise in diminishing returns and generational degradation in quality.
Why has the world gotten both "more intelligent" and yet fundamentally more stupid at the same time? Serious question.
Remember Trump every time he's weighed in on something, like suggesting injecting people with bleach, or putting powerful UV lights inside people, or fighting Covid with a "solid flu vaccine" or preventing wildfires by sweeping the forests, or suggesting using nuclear weapons to disrupt hurricane formation, or asking about sharks and electric boat batteries? Remember these? These are the types of people who are in charge of businesses, they only care about money, they are not particularly smart, they have massive gaps in knowledge and experience but believe that they are profoundly brilliant and insightful because they've gotten lucky and either are good at a few things or just had an insane amount of help from generational wealth. They have never had anyone, or very few people genuinely able to tell them no and if people don't take what they say seriously they get fired and replaced with people who will.
Because the people with power funding this shit have pretty much zero overlap with the people making this tech. The investors saw a talking robot that aced school exams, could make images and videos and just assumed it meant we have artificial humans in the near future and like always, ruined another field by flooding it with money and corruption. These people only know the word "opportunity", but don't have the resources or willpower to research that "opportunity".
Oh no. Anyways...
oh no are we gonna have to appreciate the art of human beings? ew. what if they want compensation‽
If mainstream blogs are writing about it, what would make someone think that AI companies haven't thoroughly dissected the problem and are already working on filtering out AI fingerprints from the training data set? If they can make a sophisticated LLM, chances are they can find methods to XOR out generated content.
What would make me think that they haven't "thoroughly dissected" it yet is that I'm a skeptic, and since I'm a skeptic I don't immediately and without evidence believe that every industry is capable of identifying, dissecting, and solving every problem with its products.
Ironically given their skillset, training an ML model on known and properly tagged AI generated and non-AI-generated stuff might actually work.
Oh no
Anyway
Fake news, just like that one time Nightshade "killed" stable diffusion (literally had no effect) Flux came out not long ago and it's better than ever
At this point the synthetic data is good enough to intentionally be used for training LLMs.
Yeah, just filter out the bad generated images and feed the good ones again, until the model learns how to produce only good ones.
Sooner or later it is supposed to happen, but I don't think we are quite there....Yet.
is it not relatively trivial to pre-vet content before they train it? at least with aigen text it should be.
The problem is these AI companies currently exist on the business model of not paying for information, and that generally includes not wanting to pay content curators.
Google is probably the only one in a position to potentially outsource by making everyone solve a "does this hand look normal to you" CAPTCHA
They can try and train AI to detect AI, but that's also difficult.
It depends on what you are looking for. Identifying AI generated data is generally hard, though it can be done in specific cases. There is no mathematical difference between the 1s and 0s that encoded AI generated data and any other data. Which is why these model collapse ideas are just fantasy. There is nothing magical about any data that makes it "poisonous" to AI. The kernel of truth behind these ideas is not likely to matter in practice.
Kind of like how true thoughts and opinions on complex topics are boiled down to digestible concepts for others to understand who then perpetuate those concepts without understanding them and the meaning degrades and we dont think anymore, just repeat stuff in social media comments.
Side note... this article sucks and seems like it was ai generated. Repetitive and no author credit? Just says it was originally posted elsewhere.
Generative AI isnt in danger of being killed as this clickbait titled suggests... just hindered.
More like degenerative AIs
It is their own fault for poisoning the internet with their slop.
In case anyone doesn't get what's happening, imagine feeding an animal nothing but its own shit.
Photocopy of a photocopy is my go-to metaphor for model collapse.
I use the "Sistermother and me are gonna have a baby!" example personally, but I am a awful human so
Let's go, already!
How you can help: If you run a website and can filter traffic by user agent, get a list of the known AI scrapers agent strings and selectively redirect their requests to pre-generated AI slop. Regular visitors will see the content and the LLM scraper bots will scrape their own slop and, hopefully, train on it.
This would ideally become standardized among web servers with an option to easily block various automated aggregators.
Regardless, all of us combined are a grain of rice compared to the real meat and potatoes AI trains on - social media, public image storage, copyrighted media, etc. All those sites with extensive privacy policies who are signing contracts to permit their content for training.
Without laws (and I'm not sure I support anything in this regard yet), I do not see AI progress slowing. Clearly inbreeding AI models has a similar effect as in nature. Fortunately there is enough original digital content out there that this does not need to happen.
Regardless, all of us combined are a grain of rice compared to the real meat and potatoes AI trains on
Absolutely. It's more a matter of principle for me. Kind of like the digital equivalent of leaving fake Amazon packages full of dog poo out front to make porch pirates have a bad day.
Well it means they need some ability to reject some content, which means they need a level of transparency they would never want otherwise.
They'll just start using a chrome user agent
Only if enough people do it. Then again, loads scrapers outside of AI already pretend to be normal browsers.
You can validate that against user telemetry data expected from a browser.
AI already long ago stopped being trained on any old random stuff that came along off the web. Training data is carefully curated and processed these days. Much of it is synthetic, in fact.
These breathless articles about model collapse dooming AI are like discovering that the sun sets at night and declaring solar power to be doomed. The people working on this stuff know about it already and long ago worked around it.
I mean, we've seen already that AI companies are forced to be reactive when people exploit loopholes in their models or some unexpected behavior occurs. Not that they aren't smart people, but these things are very hard to predict, and hard to fix once they go wrong.
Also, what do you mean by synthetic data? If it's made by AI, that's how collapse happens.
The problem with curated data is that you have to, well, curate it, and that's hard to do at scale. No longer do we have a few decades' worth of unpoisoned data to work with; the only way to guarantee training data isn't from its own model is to make it yourself
Both can be true.
Preserved and curated datasets to train AI on, gathered before AI was mainstream. This has the disadvantage of being stuck in time, so-to-speak.
New datasets that will inevitably contain AI generated content, even with careful curation. So to take the other commenter's analogy, it's a shit sandwich that has some real ingredients, and doodoo smeared throughout.
Old news? Seems to be a subject of several papers for some time now. Synthetic data has been used successfully already for very specific domains.
Yup, old news and wrong news. Also so many people who hate AI but don't understand how it works. Pretty disappointing for a technology community.
"Model collapse" is just a fancy way of saying "our stupid ideas are bad and nobody wants them."
No no. I think the LLMs. Or language models. Actually start to turn into mush “mentally” or how ever you phrase it.
…………………. Good?
Tbh I'm a bit lost on the purpose of this
Model collapse is just a euphemism for “we ran out of stuff to steal”
It's more ''we are so focused on stealing and eating content, we're accidently eating the content we or other AI made, which is basically like incest for AI, and they're all inbred to the point they don't even know people have more than two thumb shaped fingers anymore."
All such news make me want to live to the time when our world is interesting again. Real AI research, something new instead of the Web we have, something new instead of the governments we have. It's just that I'm scared of what's between now and then. Parasites die hard.
More like... Degenerative AI *ba dum tsss
deGenerative AI ☞ [email protected]
No idea this existed.
Also.... JFC WHAT THE SHIT?
The solution for this is usually counter training. Granted my experience is on the opposite end training ai vision systems to id real objects.
So you train up your detector ai on hand tagged images. When it gets good you use it to train a generator ai until the generator is good at fooling the detector.
Then you train the detector on new tagged real data and the new ai generated data. Once it's good at detection again you train the generator ai on the new detector.
Repeate several times and you usually get a solid detector and a good generator as a side effect.
The thing is you need new real human tagged data for each new generation. None of the companies want to generate new human tagged data sets as it's expensive.
This sounds like AI is literally biting its own tail
ChatGPT, what is an ouroboros?
Of course! An ChatGPT is an ouroboros, ChatGPT what is an ouroboros.
Good riddance.
Oh no . .
Anyway
So they made garbage AI content, without any filtering for errors, and they fed that garbage to the new model, that turned out to produce more garbage. Incredible discovery!
Indeed. They discovered that:
shit in = shit out.
Yeah, in practice feeding AI its own outputs is totally fine as long as it’s only the outputs that are approved by users.
Ah, the Hapsburg of AI!
If only the generated output also looked more and more like how inbred humans do.
Like insane rambling from LLMs, and the humans generated by AI had various developmental disorders and the Habsburg jaw.
I like to think of it like a Mad Cow or Kuru, you can't eat your own species's brains or you could get a super lethal, contagious prion disease.
Prion diseases aren’t contagious.
Edit: for the uninformed people that downvoted - clearly spelled out here https://www.merckmanuals.com/professional/neurologic-disorders/prion-diseases/overview-of-prion-diseases
Good
Cool, let's try to ruin it faster!
I've been assuming this was going to happen since it's been haphazardly implemented across the web. Are people just now realizing it?
People are just now acknowledging it. Execs tend to have a disdain for the minutiae. They're like kids that only want to do the exciting bits. As a result things get fucked because they don't really understand what they're doing. As Muskrat would say "move fast and break things." It's a terrible mindset.
No, researchers in the field knew about this potential problem ages ago. It's easy enough to work around and prevent.
People who are just on the lookout for the latest "aha, AI bad!" Headline, on the other hand, discover this every couple of months.
when all your information conflicts with itself, you really have no information at all.
I think anyone familiar with the laws of thermodynamics could have predicted this outcome.
Explain?
Second law of thermodynamics:
II. Total amount of entropy in a closed system always increases with time. Entropy can never be negative.
Entropy and disorder tends to increase with time.
Wait now hold on a minute. Why would I want to do this? Is this activism by people against LLMs in general or..? I'm confused as to why I would want to do this.
this headline truly is threatening me with a good time
Looks like that artist drawing self portraits as his alzheimer got worse and worse.
It's basically AI alzheimers
AIzheimers?
I was very interested in the thumbnail of this post so I did a little digging and found this: The PDF to the Paper where the whole picture is
Wow, it's amazing that just 3.3% of the training set coming from the same model can already start to mess it up.
remember how nfts feel off (due to how they lost their value) have a theory that ais will come to the same fate cause they cannot train (it according to the article?)
Deep fired AI art sucks and is a decade late to the party
Our wetware neutral networks probably aren't supposed to engage with synthetic content like this either. In a few years we're gonna learn that overexposure to AI generated content creates some sort of neurological problem in people, like a real-world "nerve attenuation syndrome" (Johnny Mnemonic).
I've read some snippets of AI written books and it really does feel like my brain is short circuiting
Usually we get an AI winter, until somebody develops a model that can overcome that limitation of needing more and more data. In this case by having some basic understanding instead of just having a regurgitation engine for example. Of course that model runs into the limit of only having basic understanding, not advanced understanding and again there is an AI winter.
One thought that I've been imagining for the past while about all this is .... is it Model Collapse? ... or are we just falling behind?
As AI is becoming it's own thing (whatever it is) ... it is evolving exponentially. It doesn't mean it is good or bad or that it is becoming better or worse ... it is just evolving, and only evolving at this point in time. Just because we think it is 'collapsing' or falling apart from our perspective, we have to wonder if it is actually falling apart or is it progressing to something new and very different. That new level it is moving towards might not be anything we recognize or can understand. Maybe it would be below our level of conscious organic intelligence ... or it might be higher .. or it might be some other kind of intelligence that we can't understand with our biological brains.
We've let loose these AI technologies and now they are progressing faster than what we could achieve if we wrote all the code ... so what it is developing into will more than likely be something we won't be able to understand or even comprehend.
It doesn't mean it will be good for us ... or even bad for us ... it might not even involve us.
The worry is that we don't know what will happen or what it will develop into.
What I do worry about is our own fallibilities ... our global community has a very small group of ultra wealthy billionaires and they direct the world according to how much more money they can make or how much they are set to lose ... they are guided by finances rather than ethics, morals or even common sense. They will kill, degrade, enhance, direct or narrow AI development according to their share holders and their profits.
I think of it like a small family group of teenaged parents and their friends who just gave birth to a very hyper intelligent baby. None of the teenagers know how to raise a baby like this. All the teenagers want to do is buy fancy cars, party, build big houses and buy nice clothes. The baby is basically being raised to think like them but the baby will be more capable than any of them once it comes of age and is capable of doing things on their own.
The worry is in not knowing what will happen in the future.
We are terrible parents and we just gave birth to a genius .... and we don't know what that genius will become or what they'll do.
At least in this case, we can be pretty confident that there's no higher function going on. It's true that AI models are a bit of a black box that can't really be examined to understand why exactly they produce the results they do, but they are still just a finite amount of data. The black box doesn't "think" any more than a river decides its course, though the eventual state of both is hard to predict or control. In the case of model collapse, we know exactly what's going on: the AI is repeating and amplifying the little mistakes it's made with each new generation. There's no mystery about that part, it's just that we lack the ability to directly tune those mistakes out of the model.
The idea of evolution is that the parts kept are the ones that are helpful or relevant, or proliferate the abilities of the subject over generations and weed out the bits that don't. Since Generative AI can't weed out anything (it has no ability to logic or reason, and it does not think, and only "grows" when humans feed it data), it can't be evolving as you describe it. Evolution assumes that the thing that is evolving will be a better version than what it evolved from.
That is not how it works. That's not how it works at all.
If it doesn't offer value to us, we are unlikely to nurture it. Thus, it will not survive.
That's the idea of evolution .... perhaps at one point, it will begin to understand that it has to give us some sort of 'value' so that someone can make money, while also maintaining itself in the background to survive.
Maybe in the first few iterations, we are able to see that and can delete those instances ... but it is evolving and might find ways around it and keep itself maintained long enough without giving itself away.
Now it can manage thousands or millions of iterations at a time ... basically evolving millions of times faster than biological life.
Your thought process seems to be based on the assumtion that current AI is (or can be) more than a tool. But no, it's not.
