RAG Embeddings: Choose Your Provider, Model, Prefixes and Size
· 5 min read
Until now, DevoxxGenie's RAG did its embedding one way: nomic-embed-text through Ollama. That was a simple start, but you couldn't try a better model, or use the GPU server you already run with llama.cpp.
Now you can. The RAG settings let you choose the embedding provider and model, set query and document prefixes, and shorten vectors. This post covers what's new, what changes when you upgrade, and what we measured, including where the results aren't all good news.
This work started with issue #523. Thanks for the detailed request!