2 posts tagged with "llama.cpp"

DevoxxGenie 1.18.0 is a minor release about providers. LLM routers now run on one generic provider with their own Routers settings tab, and four new routers join OpenRouter. Local OpenAI-compatible servers get the same treatment: Nativ, Rapid-MLX, GPT4All, llama.cpp and Jan now share one local provider that reads each server's real models and context window. If you're on IntelliJ IDEA 2026.3, this is also the release that makes the settings pages load again.

Until now, DevoxxGenie's RAG did its embedding one way: nomic-embed-text through Ollama. That was a simple start, but you couldn't try a better model, or use the GPU server you already run with llama.cpp.

Now you can. The RAG settings let you choose the embedding provider and model, set query and document prefixes, and shorten vectors. This post covers what's new, what changes when you upgrade, and what we measured, including where the results aren't all good news.

This work started with issue #523. Thanks for the detailed request!