3 posts tagged with "local llm"

Running a 35B model on your laptop is impressive right up until you watch it type. Local inference on Apple Silicon is memory-rich and bandwidth-poor: every single token needs its own forward pass through the whole model, and that pass is dominated by shuffling weights from unified memory into the GPU. The math is fast. The waiting is not.

MTPLX attacks exactly that bottleneck, and it does it without the usual trade-offs. On a Mac mini it lands around 1.6x faster decode; on Qwen 3.6 27B the author measures up to 2.24x. And it does that while producing the same output your model would have produced anyway.

DevoxxGenie just passed a milestone worth celebrating: more than 72,000 downloads, and in this month alone 29,330 active users firing up the plugin inside IntelliJ IDEA. Thank you. Genuinely.

A smaller, opt-in slice of those users also share anonymous usage analytics. No prompts, no code, no file contents, no personal data, just coarse aggregated signals about which features get enabled, which providers get used, and which models get picked. The goal is simple: stop guessing about what matters and start building for how people actually work.

Even from that opted-in subset, more than 18,000 telemetry events over the past month paint a remarkably clear picture. A few of the patterns surprised me. Here is the story the data tells.

Cloud APIs are convenient, but they come with per-token costs, rate limits, and the nagging question of where your code actually goes. What if you could run the same frontier-class models (GLM, MiniMax, Kimi, DeepSeek) at full Q6 or even Q8 quality, entirely on hardware you already own?

With DevoxxGenie's refreshed Exo integration, you can. Pool the memory of multiple Apple Silicon Macs into a single inference cluster and run models that would never fit on one machine. No subscription. No token budget. Just electricity.