it started with a billing problem.

i was running an off-the-shelf llm proxy to route requests between providers — the kind of thing you drop into a docker compose, point at your api keys, and hope for the best. it worked. mostly. except the usage tracking was wrong. not a little wrong — wrong wrong. like, “i thought i’d spent $12 and it was actually $47” wrong. when you’re routing traffic across xai, google, openai, deepseek, and moonshot, that kind of imprecision isn’t a rounding error. it’s a leak in the bottom of the boat.

so i built gophergate.

gophergate: a precision gooseneck kettle for api credits

think of gophergate like a gooseneck kettle — the kind coffee nerds use for pour-overs. a regular kettle dumps water everywhere. a gooseneck gives you exact control over flow, direction, and volume. that’s what gophergate does for llm api requests.

it’s a custom proxy gateway written in go. nothing fancy — go’s stdlib http server, clean routing, and an obsession with accurate token counting. every request that comes in gets routed to the right provider based on model, cost, availability, and whatever fallback logic makes sense. if openai is being flaky, it can fall back to deepseek. if you want moonshot’s pricing on a particular model, it routes there. the config is straightforward — model aliases map to provider endpoints, and the proxy handles the translation layer.

the core win is spend tracking that actually matches provider dashboards. i count input and output tokens per request, multiply by the provider’s published rates, and log it. no estimating, no “we’ll figure it out at the end of the month.” you can see exactly what each request costs in real time. when you’re running an ai coding agent that makes hundreds of calls a day, that visibility matters.

the architecture is intentionally boring:

• incoming request hits gophergate
• gophergate resolves the model alias to a provider config
• it proxies the request, streams the response back
• it logs token counts and cost
• if the provider returns an error or times out, it tries the next fallback

that’s it. no magic, no ai-powered routing decisions. just a well-tuned reverse proxy with a billing brain. go was the right call here — fast startup, tiny binary, stdlib does 90% of the work, and it compiles to a single binary i can drop anywhere on the homelab.

i’ve been running it for a few months now and the numbers are tight. gophergate’s tracked spend matches what i see on provider dashboards within a few cents. that’s the kind of boring reliability i want from infrastructure.

the agent swap: from pi.dev to hermes

for a while i was using pi.dev (formerly opencode) as my ai coding assistant. it was fine — got the job done, decent model support, the usual “type a prompt, get code” workflow. but “fine” has a shelf life, and mine ran out.

i switched to hermes agent by nous research, and the difference is less about any single feature and more about the texture of working with it. hermes doesn’t just answer questions — it builds up a persistent understanding of how you work, what you care about, and what you’ve already tried.

the skills system is the big one. skills are like procedural memory — reusable workflows that encode how to do specific tasks. i’ve got skills for deployment patterns, debugging workflows, tdd enforcement, code review. when hermes encounters a task that matches a skill, it loads the relevant knowledge and follows the proven approach instead of reinventing it every time. it’s like having a senior engineer who actually remembers what they learned last sprint.

then there’s mnemosyne — hermes’ persistent memory system. it remembers things across sessions. not just “what did we talk about yesterday” but real operational context: which services live where, what the last deployment broke, what naming conventions we use. i can come back to a project after a week and hermes picks up the thread without me re-explaining the whole setup.

cron jobs are another thing that surprised me. hermes can run scheduled tasks — monitoring, health checks, periodic reports. i’ve got a few set up to check service status and flag issues before they become problems. it’s the kind of thing you’d normally wire up with a separate tool, but having it built into the agent means the agent that understands the codebase is also the one monitoring it.

and then there’s multi-agent delegation. for complex tasks, hermes can spin up sub-agents to work on independent pieces in parallel. it’s not just “do this one thing” — it’s “here’s a plan, go execute it on multiple fronts.” the parent agent coordinates, the sub-agents execute, and the whole thing converges faster than a single-threaded approach.

replacing pi.dev wasn’t a dramatic moment. it was more like switching from a drip machine to a proper pour-over — the coffee is still coffee, but the control and consistency are on another level.

local models: the experimental corner

while the cloud providers power most of my daily work through gophergate, i’ve been poking at running smaller models locally on the homelab. this is more “weekend curiosity” than “production workflow” at this point, but it’s been fun.

the homelab has a decent gpu — nothing datacenter-grade, but enough to run quantized models in the 7b-13b range at reasonable speeds. i’ve been experimenting with a few things:

• coding assistants for quick edits — small models that can handle “rename this variable” or “add error handling here” without burning cloud credits. they’re not as good as the big models, but for simple tasks they’re surprisingly capable.
• summarization and classification — routing and categorizing logs, summarizing long threads, tagging content. these tasks don’t need frontier intelligence, and running them locally means zero latency and zero cost.
• fine-tuning experiments — i’ve been playing with lora adapters on smaller models to see if i can make them better at specific tasks in my workflow. early days, but the idea of a model that’s tuned to my codebase is compelling.

the trick is knowing where local models shine and where they don’t. they’re great at well-defined, pattern-heavy tasks. they’re not great at “architect a new feature from scratch” or “debug this subtle concurrency issue.” the cloud models through gophergate handle the hard stuff; the local models handle the grunt work.

i’ve been using ollama to manage local models — it makes pulling, running, and switching between models dead simple. combined with hermes as the orchestrator, i can have the agent decide when a task needs the big guns and when a local model will do. that’s not wired up as a formal workflow yet, but the pieces are all there.

the stack as it stands

so here’s where things land today:

gophergate sits at the edge, routing all llm traffic. it knows about every provider, tracks every dollar, and fails over gracefully. it’s the barista that knows exactly how much each pour costs.

hermes agent is the brain. it’s the thing i actually interact with — the agent with memory, skills, scheduled tasks, and the ability to delegate. it talks to the outside world through gophergate.

local models are the experimental wing. small, fast, free, good enough for a growing number of tasks. they sit on the homelab and handle what they can.

and underneath all of it, go and the homelab keep things running. no kubernetes, no terraform, no 47 microservices. just well-understood tools doing their job.

the ai tooling landscape moves fast — what’s best-in-class today is legacy tomorrow. but the principles are stable: accurate billing, persistent context, composable workflows, and knowing when a $0 local model beats a $0.02 cloud call. build for the principles and the tools can swap underneath.

now if you’ll excuse me, i need to go adjust my pour-over recipe. i think the grind is a touch too fine.

-dustin