Flash-Next Q2 on Strata, on an AMD card: how it became my daily driver
A config flag made a paid seat look like it was degrading at long context. Sorting that out is what made a 176B mixture-of-experts model on one 24 GB AMD card my daily driver.
holler is where I share wins and losses from working with AI locally: a 24 GB AMD 7900 XTX box (llama.cpp on Vulkan, fronted by llama-swap), a laptop worker, and burst cloud pods for frontier models.
What it is: incidents, A/B numbers, and the rules I added after getting burned. Every claim carries its measurement; every build carries its SHA. Take it with a grain of salt — versions drift, boxes differ, and my setup is not yours.
Why it exists: I want a running record of what actually broke, what fixed it, and what it cost. And I’m hoping other people start doing the same — posting the walls they hit and how they got over them. A pile of other people’s failures is the fastest way to not repeat them: the issue you hit at 1 a.m. is probably someone’s post from last month. If you’re running local models, post your issues too. It might be the thing that saves me next time.
Wrong or outdated? Holler back — corrections get folded in with a note.
Seven weeks of Qwen 3.8: marking the milestone with Qwen 4 on the horizon
The 3.8 base ties the strongest fine-tune on the roster within measured noise; the peak — 0.931 — is a configuration, not a download.
Turning on a speed feature dropped my hard-reasoning scores from 39/40 to 0/40
An A/B test reveals that enabling draft-MTP speculative decoding on a Q4_K_M 27B model silently collapses long-chain reasoning performance, matching an upstream bug class.
One session, 1.9M tokens, idea to launch
One continuous local-27B session took a coding-agent TUI from a closed research handoff to a launched build — long-horizon work without a frontier model or a million-token context.
A 9B I told to log a bug went and found it instead
An Ornith-1.5-9B I installed to cover a gap while the big box was down answered real infra questions in short bursts — and, told to “log a bug,” went on a root-cause hunt that nearly fixed it before the context ceiling killed the run.
The GSQ 3-bit that carried four hundred thousand tokens
One working day running the GSQ-RCO 3-bit quant of Qwen3.8-27B as the daily coding agent across eight sessions: decode held, prefill paid, five sessions overflowed the box’s cram budget, and the honest verdict — not a step down, a little looser, and I’m reverting to the 4-bit tomorrow.
The 3-bit model ran all day. It didn't break. But it kept trying to edit the same broken line.
After a week of benching, the 3-bit quant finally carried real traffic — and its quirks were weirder than the numbers suggested.
Qwen3.8's thinking knob is not a volume knob
off / low / medium / xhigh on the same 27B model, same trap prompt, temp 0 — xhigh turns out to be a near-duplicate of default, the instruction-less ‘medium’ thinks more than ’low’, and no level drifted off the right answer.
I paid $4 for a frontier model to defrag my agent's memory
Two headless dispatches to a rented 320B MoE rebuilt my coding agent’s drifted markdown memory — −21.5% standing context for about $4 of billed time — and the reusable part isn’t the defrag, it’s the dispatch contract.
Qwen3.8-27B on a 24 GB 7900 XTX: the 3-bit quant won my long-context jobs
Two quants of the same 27B dense model on a 24 GB RX 7900 XTX — Q4_K_M vs a 3-bit GSQ-RCO IQ3_S with a real MTP head. Speed is a wash past 99k tokens; the 4 GiB the smaller file frees is what decides where your context ceiling actually is.
A kill→respawn race left my 7900 XTX serving at CPU speed for 2.5 hours
llama-swap’s hot reload killed my 27B server; the respawn six seconds later landed on the dying server’s VRAM residue and llama.cpp silently fell back to CPU. Nothing errored. Here’s the failure, the A/B proof, and the 9-second probe I now run after every deploy.