Valluvan hero background
← Back to indexSelf-hosted infra

Valluvan

A Go proxy that puts Groq, OpenRouter, and OpenAI behind one endpoint, with automatic rate-limit failover.

Role

Solo infrastructure build, self-hosted

Timeline

Self-hosted infrastructure, ongoing

Stack

  • Go·
  • SQLite·
  • HTMX·
  • round-robin routing
OverviewArchitectureGallery

The problem

Any project that calls more than one LLM vendor eventually writes the same brittle code: try provider A, catch its rate limit, fall back to provider B, normalize the response shape. Rebuilding that logic inside every project is wasted effort and a reliability risk every time it's rebuilt slightly differently.

The design decision

One Go proxy sits in front of Groq, OpenRouter, and OpenAI behind a single /v1/chat/completions endpoint. Round-robin routing spreads load across providers, and automatic failover means a rate limit on one vendor doesn't stall the request — it just tries the next one. Any client that speaks the OpenAI chat-completions format works against it unmodified.

Where it stands

Self-hosted and in ongoing use as infrastructure underneath other projects — the kind of tool that's judged by whether it disappears into the background and just works.

Known rough edge

SQLite-backed state doesn't survive a reboot on ephemeral cloud storage — a real constraint for cloud deployment that needs a persistent volume mounted, documented rather than discovered the hard way in production.