LLM Providers Overview

DevoxxGenie supports a wide range of Language Model (LLM) providers, both local and cloud-based. This flexibility allows you to choose the provider that best fits your needs in terms of performance, privacy, and cost.

Supported Provider Types

DevoxxGenie integrates with two main types of LLM providers:

Local LLM Providers

Local providers run models directly on your machine, offering:

• Privacy: Your code and prompts stay on your computer
• No API costs: Use without usage fees or API keys
• Offline usage: Work without internet connection
• Customization: Fine-tune models for your specific needs

Supported local providers:

• Ollama
• LMStudio
• GPT4All
• Llama.cpp
• Jan
• Nativ (MLX on Apple Silicon)
• Exo (Distributed AI cluster)
• Custom OpenAI-compatible

Cloud LLM Providers

Cloud providers offer powerful models as a service:

• Power: Access to state-of-the-art models
• No local resources: Run without high-end hardware
• Easy setup: No model downloads or configuration
• Regular updates: Always use the latest models

Supported cloud providers:

• OpenAI (GPT-4o, O-series, etc.)
• Anthropic (Claude models)
• Google (Gemini models)
• Grok (xAI)
• Mistral
• Groq
• DeepInfra
• DeepSeek
• Kimi
• GLM (Zhipu AI)
• OpenRouter
• Cloudflare AI Gateway (one key for every provider behind your gateway)
• Azure OpenAI
• Amazon Bedrock

Provider Selection

You can select and configure LLM providers in two ways:

1. From the DevoxxGenie Panel: Use the dropdown in the top-right corner of the DevoxxGenie window
2. From Settings: Go to Settings > Tools > DevoxxGenie

Provider Comparison

Provider	Privacy	Setup Complexity	Cost	Performance	Multimodal
Local Providers
Ollama	High	Low	Free	Varies by hardware	Some models
LMStudio	High	Medium	Free	Varies by hardware	Some models
GPT4All	High	Low	Free	Basic	No
Llama.cpp	High	High	Free	Varies by hardware	Some models
Jan	High	Low	Free	Basic	No
Nativ	High	Low	Free	Optimized for Apple Silicon	Some models
Cloud Providers
OpenAI	Low	Low	Pay-per-token	Excellent	GPT-4V+
Anthropic	Low	Low	Pay-per-token	Excellent	Claude 3+
Google	Low	Low	Pay-per-token	Excellent	Yes
Grok	Low	Low	Pay-per-token	Very Good	Varies
Mistral	Low	Low	Pay-per-token	Very Good	No
Groq	Low	Low	Pay-per-token	Very Good	No
DeepSeek	Low	Low	Pay-per-token	Good	No
Kimi	Low	Low	Pay-per-token	Very Good	No
GLM	Low	Low	Pay-per-token	Very Good	No
Azure OpenAI	Medium	Medium	Pay-per-token	Excellent	Depends
Amazon Bedrock	Medium	High	Pay-per-token	Very Good	Some models

Choosing the Right Provider

For Local Providers

• Hardware Requirements: Local models need sufficient RAM and CPU/GPU
• Model Size vs. Quality: Smaller models run faster but may have lower quality
• Specific Capabilities: Some models are specialized for code, some for general text

For Cloud Providers

• Cost: Different providers have different pricing structures
• Quality Needs: Some tasks require more powerful models
• Context Window: How much context can be included with your prompts
• Specific Features: Some providers offer unique capabilities

Provider Configuration

Local Providers

• Endpoint URLs: Where to connect to the local service
• Model Selection: Which model to use
• Parameter Settings: Temperature, tokens, etc.

Cloud Providers

• API Keys: Authentication keys for the service
• Model Selection: Which model variant to use
• Parameter Settings: Temperature, tokens, etc.

For detailed configuration instructions, see:

• Local Models Configuration
• Cloud Models Configuration
• Custom Providers

Troubleshooting Provider Issues

If you encounter issues with a specific provider:

1. Check Connection: Ensure the service is running and accessible
2. Verify Settings: Double-check API keys and endpoint URLs
3. Check Logs: Look for error messages in the IDE log
4. Provider Status: Check the provider's status page for outages
5. Model Selection: Try a different model from the same provider

For more detailed troubleshooting steps, see the specific provider pages.