DIO
activeDrop-in OpenAI- and Ollama-compatible LLM gateway that learns each backend's latency online and routes vLLM / SGLang / TGI / Ollama with SLO-aware admission. No engine patches.
Code-only MCP server: no hosted endpoint, so you install and run it yourself. It is not probed, graded or counted in our reachability statistics.
Get it
- Repository
- github.com/nisaral/DIO
- License
- Apache-2.0
- Last update
- 2026-11-01
- GitHub stars
- ★ 3
- Homepage
- doi.org/10.5281/zenodo.22085398
Listed in
- github · source
Run this server as a hosted endpoint? Submit the endpoint and it moves to the remote directory, where it is probed and graded.