[Paper] LLM ASICs on USB sticks?
[Paper] LLM ASICs on USB sticks?
cross-posted from: https://lemmy.ml/post/16728823
Source: nostr
Building intelligent robots that can converse with us like humans requires massive language models that can process vast amounts of data. However, these models rely heavily on a mathematical operation called Matrix multiplication (MatMul), which becomes a major bottleneck as the models grow in size and complexity. The issue is that MatMul operations consume a lot of computational power and memory, making it challenging to deploy these robots in smaller, more efficient bodies. But what if we could eliminate MatMul from the equation without sacrificing performance? Researchers have made a breakthrough in achieving just that, creating models that are just as effective but use significantly less energy and resources. This innovation has significant implications for the development of embodied AI companions, as it brings us closer to creating robots that can think and learn like humans while running on smaller, more efficient systems. This could lead to robots that can assist us in our daily lives without being tethered to a power source.
by Llama 3 70B
Other work:
- The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits (from February 2024)
- Ternary Neural Networks for Resource-Efficient AI Applications (2016)
I think that also did away with the matrix multiplications.