Technology @lemmy.world Michael Ten @lemmy.world 8mo ago

Opera is testing letting you download LLMs for local use, a first for a major browser

www.zdnet.com Opera is testing letting you download LLMs for local use, a first for a major browser

You can now use LLMs in Opera locally, meaning without having to send information to a server.

AI Companions @lemmy.world pavnilschanda @lemmy.world 8mo ago

News

Opera is testing letting you download LLMs for local use, a first for a major browser

www.zdnet.com /article/opera-is-testing-letting-you-download-llms-for-local-use-a-first-for-a-major-browser/

0 0

AI @lemmy.ml Michael Ten @lemmy.world 8mo ago

Opera is testing letting you download LLMs for local use, a first for a major browser

www.zdnet.com /article/opera-is-testing-letting-you-download-llms-for-local-use-a-first-for-a-major-browser/

12 2

39 comments

I haven't used Opera since they switched from their own engine to chrome. They are now owned by a Chinese company, so it probably has at least as much tracking built into it as Google Chrome now.
- I miss old opera before the buyout
  
  That's essentially Vivaldi now.
  
  Vivaldi is made by many of the same people with similar features and vibe. It's also chromium-based, though.
- I used Fifth a bit, which is something aesthetically similar to old Opera made with fltk and a webkit port to fltk. But it's abandoned now.
  
  It's so sad really, when I was a Windows user, it was Opera, when I moved to Linux, it was again Opera, then I also started using Conkeror (based on XULRunner).
  
  Then Opera died. Then XULRunner died. No usable web browser anymore.
  
  Don't you like Firefox? It's on both win and Linux.
Why the hell do I need this in a web browser? Why isn’t it a stand alone app?
- If you think of LLMs as a thing to replace search bars then this kind of makes sense.
  
  Just more unnecessary browser bloat.
  
  If you think of LLMs as a thing to replace search bars
  
  I don't.
- There are plenty of stand-alone LLM apps.
- Same reason people get their WiFi from their ISP Modem+Router combo, even though it's stupid to do so: People often confuse initial convenience for good.
Thats a cool feature for sure but I don't trust opera.
https://rentry.co/operagx
Can't they just stick to normal browser things like gaming integrations?
Intresting. But I'm curious about the performance.

A bigger LLM (mixtral) already struggles to run on my mid-range gaming PC. Trying to run an LLM that isn't terrible on a standard laptop wouldn't be a good experience.
- I have no idea how this is set up to work technically, but most of the heavy lifting is gonna be on the GPU. I'm not sure that it matters much whether the browser is what's pushing data to the GPU or some other package.
  
  Most people probably don't have a dedicated GPU and an iGPU is probably not powerfull enough to run an LLM at decent speed. Also a decent model requires like 20GB of RAM which most people don't have.
  
  Unlikely, at least on non-nvidia chips, and even on AMD, it's only the latest four chips that support it. Anything older isn't going to cut it.
  
  You also need a fairly big amount of VRAM for models like that. (4 GB is the minimum for the common kinds, which is more than typical integrated systems, or 8 GB of system memory). You can get by with system RAM, but the performance will be quite bad, since you're either relying on the CPU, or you'll be adding the latency from data moving between them.
  
  The thought that internet becomes shitty enough that you need a GPU to browse it is really frightening me. If we really reach that point that may be to run an AI which filters out AI generated spam which would really depress me 😭
What's LLM?
- https://en.wikipedia.org/wiki/Large_language_model
  
  A lot of the "AI" stuff that's been in the news recently, chatbots and image generation and such, are based on LLMs.

39 comments