OpenAI Furious DeepSeek Might Have Stolen All the Data OpenAI Stole From Us
OpenAI Furious DeepSeek Might Have Stolen All the Data OpenAI Stole From Us
OpenAI shocked that an AI company would train on someone else's data without permission or compensation.
The venture capitalist and new Trump administration member David Sacks, meanwhile, said that there is “substantial evidence” that DeepSeek “distilled the knowledge out of OpenAI’s models.”
“There’s a technique in AI called distillation, which you’re going to hear a lot about, and it’s when one model learns from another model, effectively what happens is that the student model asks the parent model a lot of questions, just like a human would learn, but AIs can do this asking millions of questions, and they can essentially mimic the reasoning process they learn from the parent model and they can kind of suck the knowledge of the parent model,” Sacks told Fox News. “There’s substantial evidence that what DeepSeek did here is they distilled the knowledge out of OpenAI’s models and I don’t think OpenAI is very happy about this.”
This sounds like horse shit to me but I don't know the technical details well enough to say with confidence.
also "suck the knowledge out of the parent model" what the actual fuck?
Footage of Deepseek slurping the knowledge out of the GPT4
I don't know enough to say whether this is valid or just crybaby tech bros having a fit on fox news but like... God I hope deepseek is completely stolen like this because and I hope there's absolutely nothing closedai can do about the fact that there's a better thief out there on the market. Fuck them so hard and fuck their hypocrisy about stealing data. Maybe we can finally move away from trying to use a double digit percentage of national electric grid capacity to power a fucking glorified magic 8ball
This is much more a TechTakes story than a NotAwfulTech one; let's keep the discussion over on the other thread:
Noted going forward. Sorry about that! ❤
They don't have any evidence. They say someone did "hammer their API", and then they terminated their license (last year), but they don't know who. China bashing is not going to depend on actual evidence.
All that matters, in the end, is "customer prices" instead of our devoted love for Sam Altman.
Knowledge distilation is training a smaller model to mimic the outputs of a larger model. You don't need to use the same training set that was used to train the larger model (the whole internet or whatever they used for chatgpt), but can use a transfer set.
Here's a reference: Hinton, Geoffrey. "Distilling the Knowledge in a Neural Network." arXiv preprint arXiv:1503.02531 (2015)., https://arxiv.org/pdf/1503.02531