/holler/Field notes from a small local-inference fleet: AMD 7900 XTX on Vulkan, llama.cpp, llama-swap, and burst cloud pods for frontier models. Measured, not vibes.en-usSun, 11 Oct 2026 21:40:10 +0000/holler/posts/flashnext-q2-strata-amd-daily-driver//holler/posts/flashnext-q2-strata-amd-daily-driver/A config flag made a paid seat look like it was degrading at long context. Sorting that out is what made a 176B mixture-of-experts model on one 24 GB AMD card my daily driver.Sun, 11 Oct 2026 00:00:00 +0000strataamdrdna3flash-nextqwenquantizationcontextlocal-llm/holler/posts/qwen38-seven-weeks//holler/posts/qwen38-seven-weeks/The 3.8 base ties the strongest fine-tune on the roster within measured noise; the peak — 0.931 — is a configuration, not a download.Fri, 25 Sep 2026 00:00:00 +0000qwen3.8qwen4llama.cppspec-decodequantizationfine-tune/holler/posts/spec-decode-quantized-targets-degradation//holler/posts/spec-decode-quantized-targets-degradation/An A/B test reveals that enabling draft-MTP speculative decoding on a Q4_K_M 27B model silently collapses long-chain reasoning performance, matching an upstream bug class.Wed, 16 Sep 2026 00:00:00 +0000llama.cppspeculative-decodingquantizationquality-evaluation/holler/posts/qwen38-27b-long-session-launch//holler/posts/qwen38-27b-long-session-launch/One continuous local-27B session took a coding-agent TUI from a closed research handoff to a launched build — long-horizon work without a frontier model or a million-token context.Sat, 12 Sep 2026 00:00:00 +0000local-llmqwenagent-harnesslong-contextcompaction/holler/posts/ornith-9b-bug-hunt//holler/posts/ornith-9b-bug-hunt/An Ornith-1.5-9B I installed to cover a gap while the big box was down answered real infra questions in short bursts — and, told to “log a bug,” went on a root-cause hunt that nearly fixed it before the context ceiling killed the run.Fri, 11 Sep 2026 00:00:00 +0000ornithagentic-rlllama-cppinference-laptopcontext-window/holler/posts/qwen38-gsq-workday//holler/posts/qwen38-gsq-workday/One working day running the GSQ-RCO 3-bit quant of Qwen3.8-27B as the daily coding agent across eight sessions: decode held, prefill paid, five sessions overflowed the box’s cram budget, and the honest verdict — not a step down, a little looser, and I’m reverting to the 4-bit tomorrow.Thu, 10 Sep 2026 00:00:00 +0000llama.cppqwenquantizationgsqllama-swapamdcoding-agent/holler/posts/memory-defrag-dispatch//holler/posts/memory-defrag-dispatch/Two headless dispatches to a rented 320B MoE rebuilt my coding agent’s drifted markdown memory — −21.5% standing context for about $4 of billed time — and the reusable part isn’t the defrag, it’s the dispatch contract.Mon, 07 Sep 2026 00:00:00 +0000agentsmemoryheadless-dispatchrunpod/holler/posts/qwen38-thinking-levels//holler/posts/qwen38-thinking-levels/off / low / medium / xhigh on the same 27B model, same trap prompt, temp 0 — xhigh turns out to be a near-duplicate of default, the instruction-less ‘medium’ thinks more than ’low’, and no level drifted off the right answer.Mon, 07 Sep 2026 00:00:00 +0000llama.cppqwenreasoningthinking-modelsamdrdna3/holler/posts/qwen38-gsq-production-day//holler/posts/qwen38-gsq-production-day/After a week of benching, the 3-bit quant finally carried real traffic — and its quirks were weirder than the numbers suggested.Mon, 07 Sep 2026 00:00:00 +0000llama.cppqwenquantizationllama-swapamd/holler/posts/xtx-cpu-fallback-incident//holler/posts/xtx-cpu-fallback-incident/llama-swap’s hot reload killed my 27B server; the respawn six seconds later landed on the dying server’s VRAM residue and llama.cpp silently fell back to CPU. Nothing errored. Here’s the failure, the A/B proof, and the 9-second probe I now run after every deploy.Sun, 06 Sep 2026 00:00:00 +0000llama.cppllama-swapvulkanamdrdna3troubleshooting