LLM Providers Overview
DevoxxGenie supports a wide range of Language Model (LLM) providers, both local and cloud-based. This flexibility allows you to choose the provider that best fits your needs in terms of performance, privacy, and cost.
Supported Provider Types
DevoxxGenie integrates with two main types of LLM providers:
Local LLM Providers
Local providers run models directly on your machine, offering:
- Privacy: Your code and prompts stay on your computer
- No API costs: Use without usage fees or API keys
- Offline usage: Work without internet connection
- Customization: Fine-tune models for your specific needs
Supported local providers:
Cloud LLM Providers
Cloud providers offer powerful models as a service:
- Power: Access to state-of-the-art models
- No local resources: Run without high-end hardware
- Easy setup: No model downloads or configuration
- Regular updates: Always use the latest models
Supported cloud providers:
- OpenAI (GPT-4o, O-series, etc.)
- Anthropic (Claude models)
- Google (Gemini models)
- Grok (xAI)
- Mistral
- Groq
- DeepInfra
- DeepSeek
- Kimi
- GLM (Zhipu AI)
- OpenRouter
- Azure OpenAI
- Amazon Bedrock
Provider Selection
You can select and configure LLM providers in two ways:
- From the DevoxxGenie Panel: Use the dropdown in the top-right corner of the DevoxxGenie window
- From Settings: Go to Settings > Tools > DevoxxGenie
Provider Comparison
| Provider | Privacy | Setup Complexity | Cost | Performance | Multimodal |
|---|---|---|---|---|---|
| Local Providers | |||||
| Ollama | High | Low | Free | Varies by hardware | Some models |
| LMStudio | High | Medium | Free | Varies by hardware | Some models |
| GPT4All | High | Low | Free | Basic | No |
| Llama.cpp | High | High | Free | Varies by hardware | Some models |
| Jan | High | Low | Free | Basic | No |
| Cloud Providers | |||||
| OpenAI | Low | Low | Pay-per-token | Excellent | GPT-4V+ |
| Anthropic | Low | Low | Pay-per-token | Excellent | Claude 3+ |
| Low | Low | Pay-per-token | Excellent | Yes | |
| Grok | Low | Low | Pay-per-token | Very Good | Varies |
| Mistral | Low | Low | Pay-per-token | Very Good | No |
| Groq | Low | Low | Pay-per-token | Very Good | No |
| DeepSeek | Low | Low | Pay-per-token | Good | No |
| Kimi | Low | Low | Pay-per-token | Very Good | No |
| GLM | Low | Low | Pay-per-token | Very Good | No |
| Azure OpenAI | Medium | Medium | Pay-per-token | Excellent | Depends |
| Amazon Bedrock | Medium | High | Pay-per-token | Very Good | Some models |
Choosing the Right Provider
For Local Providers
- Hardware Requirements: Local models need sufficient RAM and CPU/GPU
- Model Size vs. Quality: Smaller models run faster but may have lower quality
- Specific Capabilities: Some models are specialized for code, some for general text
For Cloud Providers
- Cost: Different providers have different pricing structures
- Quality Needs: Some tasks require more powerful models
- Context Window: How much context can be included with your prompts
- Specific Features: Some providers offer unique capabilities
Provider Configuration
Local Providers
- Endpoint URLs: Where to connect to the local service
- Model Selection: Which model to use
- Parameter Settings: Temperature, tokens, etc.
Cloud Providers
- API Keys: Authentication keys for the service
- Model Selection: Which model variant to use
- Parameter Settings: Temperature, tokens, etc.
For detailed configuration instructions, see:
Troubleshooting Provider Issues
If you encounter issues with a specific provider:
- Check Connection: Ensure the service is running and accessible
- Verify Settings: Double-check API keys and endpoint URLs
- Check Logs: Look for error messages in the IDE log
- Provider Status: Check the provider's status page for outages
- Model Selection: Try a different model from the same provider
For more detailed troubleshooting steps, see the specific provider pages.