All writing
Local inference, AI infrastructure, MSO and telecom architecture, and field notes from the systems I build and the work I do.
Taalas physically embodied Llama 3.1 8B in silicon, reached roughly 17,000 tokens per second for one user, and was acquired by AMD before I finished writing about it.
Read →ROCm worked on the first try. Vulkan was substantially faster, and the hardest failures belonged to everything around the card.
Read →Five mapped expert layers made room for speculative decoding, a correction tier, and near-million-token context.
Read →The origin story, hardware escalation, runtime experiments, failures, and corrections behind a private local AI system.
Read →