Exploring Llamafile: Mozilla's Attempt in the World of Open Source AI
Exploring Llamafile: Mozilla's Attempt in the World of Open Source AI
itsfoss.com Exploring Llamafile: Mozilla's Attempt in the World of Open Source AI
Now Mozilla tries to make running LLMs easier with Llamafile project. Here's my experience with it.
You're viewing a single thread.
4 comments
The issue isnt if its easy to use the issue is a matter of compute. A majority of people on the internet only have mobile access. We need a way to let the masses utilise distributed compute in a secure and private way.
You can run many 7B models on phones with 8GB RAM
I find the 7b models dont have the capabilities id like and i cant image the tokens per second would be very good.
Yeah running it and it actually being useful are two very different things.