4 posts tagged with "qwen"

The local AI story on the Mac has always been a bit of a scavenger hunt. The models live in a Hugging Face cache, the server is a Python process you babysit in a terminal, the performance numbers are whatever flew past in the logs, and the chat UI is a browser tab pointed at localhost. It all works, but nothing about it feels like it belongs on a Mac.

Nativ fixes that with one opinionated move: it puts the whole workflow — chat, model management, serving, monitoring, and even hardware telemetry — inside a single native SwiftUI app. And as of DevoxxGenie 1.10.1, it's a first-class provider in the plugin, sitting right next to Ollama and LM Studio in the LLM Providers list.

Running a 35B model on your laptop is impressive right up until you watch it type. Local inference on Apple Silicon is memory-rich and bandwidth-poor: every single token needs its own forward pass through the whole model, and that pass is dominated by shuffling weights from unified memory into the GPU. The math is fast. The waiting is not.

MTPLX attacks exactly that bottleneck, and it does it without the usual trade-offs. On a Mac mini it lands around 1.6x faster decode; on Qwen 3.6 27B the author measures up to 2.24x. And it does that while producing the same output your model would have produced anyway.

Here's a quietly wild fact: NVIDIA is casually giving you access to a whole catalogue of frontier AI models - including five of the strongest Chinese frontier models - for free. 😳

No credit card. No subscriptions. Just one API key that unlocks everything.

Starting with DevoxxGenie 1.8.11, the plugin ships NVIDIA as a first-class cloud provider. Drop in your nvapi-... key, and the entire NVIDIA build catalogue shows up in the model dropdown - over 100 models, all reachable through a single OpenAI-compatible endpoint.

DevoxxGenie just passed a milestone worth celebrating: more than 72,000 downloads, and in this month alone 29,330 active users firing up the plugin inside IntelliJ IDEA. Thank you. Genuinely.

A smaller, opt-in slice of those users also share anonymous usage analytics. No prompts, no code, no file contents, no personal data, just coarse aggregated signals about which features get enabled, which providers get used, and which models get picked. The goal is simple: stop guessing about what matters and start building for how people actually work.

Even from that opted-in subset, more than 18,000 telemetry events over the past month paint a remarkably clear picture. A few of the patterns surprised me. Here is the story the data tells.