Architecture

Everything in hlid serves one job: handle one request. Trace a request through, and the components fall out.

 Agent                                                      Backend
  │  POST /v1/messages (a dialect)                         (llama/vLLM/cloud)
  ▼                                                              ▲
[ingress] → [auth] → [router] ──lookup──→ registry(model→backend+dialect+key)
                        │                                        │
                        ▼                                        │
                 dialect match? ── yes ──> [proxy passthrough] ──┘   (common case)
                        │                                        │
                        └── no ──> [translate] ──> [upstream] ───┘   (cross-dialect only)
                        response ◄── (translate back if needed) ◄── stream ◄────
                        └──────────── [observe] captures the whole trip ────────

Passthrough-first

Most traffic needs zero translation: llama.cpp speaks every dialect an agent does, and same-vendor pairs (OpenAI agent → OpenAI backend, Anthropic agent → Anthropic backend) are byte passthrough by definition. hlid never materializes the body on that path — bytes stream through untouched.

Real translation is only needed cross-dialect, which collapses to one boundary: OpenAI-family ⇄ Anthropic-family. There is no canonical internal format that every request pays for — that's the tax hlid refuses.

Components

Module	One job
ingress	Accept requests: axum routes, body limits, CORS, timeouts, middleware
auth	Is this caller allowed in? Constant-time key check; loopback bypass
router	Model name → backend (url, dialect, credential) via the registry
proxy	Passthrough: forward, inject credentials, stream back unchanged
translate/	Cross-dialect conversion — requests, responses, and streaming SSE
upstream	reqwest client: forward, bounded retry on cold-start 503, backpressure
observe	Capture each trip into a bounded ring buffer; serve /hlid/requests
config	Load file + env; validate the registry at startup
error	Typed errors → dialect-native error bodies

Stack

• tokio — async runtime
• axum (on hyper) — HTTP server and routing
• reqwest — upstream client, streaming
• serde / serde_json — typed dialect (de)serialization; translation is compile-time-checked struct-to-struct mapping
• tower / tower-http — middleware (auth, limits, CORS, timeouts)
• tracing — structured logs

Testing philosophy

The test suite (205 tests) is the spec:

• Golden tests — recorded fixtures (baseline/, tools/, degraded/) pin exact translation output, including streaming SSE event sequences
• E2E pipeline tests — full trips against mock backends, passthrough and cross-dialect, streaming and not
• Unit tests per module — auth, registry resolution, retry behavior, ring-buffer eviction, header filtering

If you're changing translation behavior, start by adding a fixture that shows the output you expect — then make it pass.

Design decisions

• Route / translate / observe only. No model lifecycle — hlid never starts, downloads, or sizes model servers.
• Unified surface: Responses + Messages feature-full; Chat + legacy Completions for compatibility.
• Passthrough-first; translate only where forced. Build only the directions real traffic needs.
• Observability is first-class, not an add-on.