They say no llama.cpp support yet, so no ollama yet (which uses llama.cpp)

HanClinto · on July 16, 2024

Correct. The only back-end that Ollama uses is llama.cpp, and llama.cpp does not yet have Mamba2 support. The issues to track Mamba2 and Codestral Mamba support are here:

https://github.com/ggerganov/llama.cpp/issues/8519

https://github.com/ggerganov/llama.cpp/issues/7727

Mamba support was added in March of this year:

https://github.com/ggerganov/llama.cpp/pull/5328

I have not yet seen a PR to address Mamba2.

sadeshmukh · on July 16, 2024

Ollama is supported: https://docs.continue.dev/setup/select-provider

trsohmers · on July 16, 2024

They meant that there is no support for Codestral Mamba for llama.cpp yet.