Bonsai 27B (1-bit LLM): The First 27B-Class Model to Run on a Phone

What's the hiring space and business strategy around all of these smaller AI labs? Its really cool that people like these guys get paid to optimize models and give them out for free (open source). Do a lot of these labs have forward deployed engineers doing integrations with customers who want local models? Is there a general shift towards the local model crowd?

The KV-cache memory usage also seems remarkably frugal, even at the full context length. That could make this model particularly useful in multi-agent coding workflows.

I wish KV-cache memory usage and related optimizations were discussed more clearly in new model announcements and demos.

The models themselves are showing up on Hugging Face here: https://huggingface.co/prism-ml/models

Apparently Apple is "in talks" with the PrismML: https://www.cnbc.com/2026/07/14/apple-prismml-ai-compression...

The problem, of course, is if you run the UD_Q2 variant (Unsloth) which does only post-training, the number is pretty close to 1-bit model here and the 5% drop in tool-call is significant than it suggests in real-life use cases.

For those curious about their demo, I’m pretty sure it’s using Locally AI (iOS only) that lmstudio acquired/aquihired a couple months ago.

Tried it on Android and got "!!!!!!!!!!!!!" for answers.

I don’t know if the llama cpp implementation is wonky (and only supports the binary version) but it’s a lot slower than 35B-A3B @ Q4_KM + MTP with CPU offloading.

I've been watching and waiting for this, interested to see how smart it is, as it fits with my interest of getting the smartest possible model running in 10GB of VRAM (RTX3060 that has to drive 2 monitors and run an llm)

TIL that 1 bit models are actually 1.58 bit with three values +1, 0 and -1

I was trying Ornith 9B locally (it's up on Ollama) which claims:

> Ornith-1.0-9B, which can be easily deployed on edge devices, matches or exceeds the performance of much larger models such as Gemma 4-31B and Qwen 3.6 35B.

https://deep-reinforce.com/ornith_1_0.html

Only tried it so much so far; it did a little better than Qwen 9B

That's awesome. What's the largest model that could fit onto a single 16gb gpu at 1.125 effects bits per weight?

This must be some sort of unpublished app?

I can just see their image tool on the app store