Mistral 7B v0.2 Base (released at SHACK15sf hackathon)
Mistral 7B v0.2 Base (released at SHACK15sf hackathon)
github.com /mistralai-sf24/hackathon
GitHub: https://github.com/mistralai-sf24/hackathon
X: https://twitter.com/MistralAILabs/status/1771670765521281370
New release: Mistral 7B v0.2 Base (Raw pretrained model used to train Mistral-7B-Instruct-v0.2)
๐ธ https://models.mistralcdn.com/mistral-7b-v0-2/mistral-7B-v0.2.tar
๐ธ 32k context window
๐ธ Rope Theta = 1e6
๐ธ No sliding window
๐ธ How to fine-tune:
1 comments
It does a little bit worse than v0.1 on all benchmarks which isn't ideal. That doesn't really say much about the finetuning potential though.
2 0 Reply