MCP servers that ship as code — a repository or a package you install and run locally (usually over stdio). They have no hosted endpoint, so we cannot probe or grade them: instead each one shows its repository, install command, license, stars and when it was last updated.
Collected from GitHub repositories and npm, PyPI, Cargo, Go, Maven, NuGet and RubyGems packages tagged as MCP servers, the official MCP Registry, Docker's MCP catalog, ModelScope, Baidu MCP World, Cline and other public directories; one entry per repository. A server that also offers a hosted endpoint is listed under Remote instead. These entries are not counted in our reachability, verification or readiness statistics.
-
agent-eval-mcpgithub.com/nicolaemorcov/agent-eval-mcpupdated this year
Deterministic evaluation tools for AI coding agents, exposed as an MCP server.
pypi MIT -
kryve-agent-eval-mcpgithub.com/louislaurent1/kryve-agent-eval-mcpupdated this year
Open-source MCP server for designing and scoring practical AI-agent evaluations.
MIT -
Governance-classifier eval harness on the Claude Messages API, with calibrated LLM-as-judge and an MCP server
MIT -
NIA AI Web Evaluation Agent MCP Server - Autonomous browser testing and debugging
npm MIT -
A multi-agent system (planner, executor, critic) with an MCP tool server, a rule-based guard, and an evaluation harness.
-
MCP eval-gate server: an agent's only path to moving money runs a fail-closed, version-pinned OFAC screen-before-transfer control first. Stdlib-only, zero-install, with tests for each failure case.
MIT -
agent-eval-mcp-servergithub.com/mertdurukan/agent-eval-mcp-serverupdated this year
Production-grade MCP server that exposes AI agent quality evaluation (RAGAS faithfulness, precision, recall, response relevancy + custom JSON trace eval).
MIT -
A deployment layer for a guardrailed clinical conversational agent: real FHIR over MCP, PHI redaction, a hash-chained audit trail, and an evaluation harness measured on held-out data it was never tuned on.
MIT -
Scenario-driven framework for evaluating demographic bias in single- and multi-agent LLM workflows. Tracks bias across sequential stages, separates model non-determinism from systematic effects, and prevents agents inferring evaluation intent. REST API and MCP server.
MIT -
mcp-bigquery-evalsgithub.com/Umarfarook1/mcp-bigquery-evalsupdated this year
Read-only BigQuery MCP server with mandatory dry-run cost caps, agent-friendly structured errors, and a Spider/BIRD-style NL-to-SQL eval harness.
pypi MIT ★ 1 -
eval-integritygithub.com/ipezygj/evalgateupdated this year
Dependency-free statistical checks for AI eval claims (multiple-comparisons, judge-bias, resolution, fragility) — as a CLI and an MCP server an agent calls before trusting a benchmark number.
pypi MIT -
evals-mcp-servergithub.com/cyanheads/evals-mcp-serverno update data
Agent-driven eval authoring via a new authoring-loop archetype — draft → subagent-review → surgical-revise → submit, with server-enforced self-consistent graders; compiles to standard harnesses.
smithery -
Energy invoice extraction and reconciliation agent: evals, an MCP tool surface and tenant-isolated guardrails
-
A safety layer for AI agents that spend money: payment gate, signed-approval verifier, MCP server and evals.
MIT -
MCP server for Table Mountain weather, with an agent eval suite
MIT -
kensa-mcpgithub.com/satyaborg/kensaupdated this year
MCP server for kensa, the open source agent evals harness
pypi MIT -
The-Gate-VPSgithub.com/EditorialOS/The-Gate-VPSupdated this year
Production deployment layer for The Gate. Multi-tenant MCP server exposing the eval engine as agent-callable tools — scored eval harness, draft→eval→revise agent example, PRD, runbook, Docker.
NOASSERTION -
mediassist-aigithub.com/thewhyman/mediassist-aiupdated this year
Agentic Medicaid eligibility copilot for FQHC caseworkers — MCP + Mem0 + eval-gated deployment
-
skillsgithub.com/sattva2020/skillsupdated this year
Agent skills and MCP servers for AI-assisted development: harness audit (context, tools, memory, evals, guardrails), Dokploy API MCP
★ 3 -
Read-only MCP server with two agents that review a running week and plan the next one. All arithmetic in code, with field-based evals.
MIT -
IT/customer-support agent: MCP tool server + LangGraph, human approval for risky actions, prompt-injection & PII-leak evals. Local open-weight models (Ollama).
-
Eval-driven SQL reliability for AI agents — generate, validate, safely execute, and regression-test LLM SQL. MCP + FastAPI gateways, execution-truth evals, CI regression gate.
pypi MIT ★ 3 -
Production-shaped tool-calling agent in TypeScript: 3-point RBAC, proposal-based writes, circuit-breaker router, metering, MCP server, 40-case deterministic evals
MIT -
AI customer-support agent with tool use, human-in-the-loop approvals, guardrails, MCP server and evals — Python, FastAPI, SQLite, Claude
MIT -
agents-AIgithub.com/RenanMiqueloti/agents-AIupdated this year
Agentes LangGraph com HITL via interrupt(), MCP server custom e evals LLM-as-judge — multi-provider (Claude, OpenAI, Ollama)
MIT -
Open-source evals for agentic analytics: test your AI analytics agent's accuracy and provenance over its own MCP server.
MIT ★ 9 -
Agente de atendimento para delivery: function calling (Gemini), RAG com embeddings, servidor MCP e evals que julgam cada atendimento pelo efeito no sistema.
-
pagelensgithub.com/AlexBoghh/pagelensupdated this year
MCP server that audits how legible a web page is to AI agents, with evidence-grounded findings and evals
MIT -
Permission-aware MCP server and agent harness with role-scoped tools, whole-plan validation, deterministic safety evals, and a React playground.
MIT -
Production-minded AI procurement agent with deterministic policy gates, human approval, evals, MCP, OpenTelemetry and PostgreSQL.
-
Ghillie: a rota-keeping agent for a Scottish games circuit, built three ways on one MCP server and measured live against the same evals
MIT -
Governed AI agents with an MCP server: least-privilege tool grants, human approval for writes, PII redaction, audit log, rollback, and safety evals in CI.
MIT ★ 1 -
Agentic supply-execution copilot: MCP + LangGraph + evals, on synthetic data
MIT -
MCP server exposing multivon-eval + pdfhell as agent-callable tools. Drop into Claude Desktop, Cursor, Cline.
pypi NOASSERTION -
coach-kernelgithub.com/LeesungDang/coach-kernelupdated this year
Deterministic safety kernel for AI-agent training plans, with MCP data boundaries, Claude Agent SDK integration, replay evals, and fuzzing.
MIT -
MCP server for the Rutgers Schedule of Classes, a Claude agent that uses it, and a 20-task eval (20/20)
-
Multi-agent workflow engine — LangGraph router, domain agents, MCP tools, human-in-the-loop gates for financial actions, compensating sagas and an eval harness, built with FastAPI and Python.
MIT -
Agent control plane for governed AI coding: validate changes, enforce policy gates, track findings, proofs, and evals based on your habits.
npm NOASSERTION ★ 11 -
AI on-call copilot: hybrid RAG with citations, agent tools over telemetry, MCP server, CI-gated evals. NestJS, Next.js, Vercel AI SDK, pgvector.
MIT -
MCP server exposing live F1 game UDP telemetry to LLM agents, with evals (TypeScript)
MIT -
Local AI agent for KiCad PCB designs — netlist/schematic parser, evidence-grounded audit, deterministic sizing/BOM, read-only MCP, firmware templates (STM32 HAL/Arduino), ReAct agent (Ollama), evals
MIT -
Reliable agent tool calling on small models: schema-validated decisions, tiered routing with evidence-based escalation, an MCP server and a CI eval gate. Python stdlib only.
-
copilot-studio-simulatorgithub.com/butelo/copilot-studio-simulatorupdated this year
Unofficial local Copilot Studio simulator: declarative agents (topics, tools, MCP, code execution, knowledge, child agents) + evals + auto-tuning, built on the pi harness SDK
MIT ★ 1 -
Agente de IA con Claude que opera un servidor MCP de finanzas personales: human-in-the-loop, defensa contra prompt injection, evals automatizados en CI y trazas de costo/latencia.
-
Agentic reconciliation-break triage system — bounded LangGraph state machine, MCP trust-boundary tool split, human-approval interrupt, and a real eval harness.
★ 1 -
Business AI agent in TypeScript: Hebrew RAG, web search, and approval-gated actions (booking, email), with an MCP server and a 20-scenario eval suite
MIT -
Python framework + local Studio for reliable LLM workflows: typed files in your repo, pre-run checks, experiments and evals, MCP server for coding agents
pypi NOASSERTION ★ 4 -
supabase-realtime-skillgithub.com/0xquinto/supabase-realtime-skillupdated this year
Agent Skill + MCP server for Supabase Realtime/CDC: bounded subscription primitive that fits Edge Function isolate budgets, pgvector composition, eval-first methodology with pre-registered thresholds.
npm Apache-2.0 -
oncall-copilotgithub.com/sasi-nemani/oncall-copilotupdated this year
Provider-agnostic AI on-call assistant — RAG + a tool-using agent + an MCP server + an eval harness — runs on Claude, OpenAI or OpenRouter. A hands-on learning project.
GPL-3.0 ★ 1 -
SuperBryn MCP server — run the voice-agent eval loop and read your production call logs from Claude Code, Claude Desktop, or Codex.
npm MIT -
@myth-os/mcpgithub.com/starhub-studios/mythos-mcpupdated this year
MCP server for narrative identity, governance, eval quality, and memory. Gives your AI agent a persistent understanding of who it's working for.
npm MIT -
RAG agent + MCP server over the ROS 2 Jazzy docs, with cited answers and an eval suite
-
AI-optional accounts payable agent: deterministic financial core, stateless MCP server, bilingual Next.js UI, and offline evals that run with no API key.
MIT -
Enterprise agentic IT/Ops support triage: a multi-step LLM agent with an MCP server, RBAC + human-approval gates, hash-chained audit, RAG with data governance, and an eval-gated CI pipeline. Python/FastAPI + React/TS; runs fully offline.
MIT -
A Model Context Protocol server that lets an agent eval C# inside a live .NET process through Sonde.
nuget MIT -
PokéDex con IA: agente con tool calling escrito a mano, servidor MCP, 3 LLMs con fallback (Groq → Gemini → Anthropic), evals y visión. Next.js + Supabase (RLS) + FastAPI.
MIT -
invariantevalgithub.com/AlpharomeroJL/invariantevalupdated this year
Declared safety-invariant eval gate: enforces your locked-field rules on every output and refuses to finalize when a model auto-fills a human-owned field. MCP tool and agent skill included.
MIT ★ 1 -
numguardgithub.com/ipezygj/numguardupdated this year
The verification layer for the agent economy: an agent-callable primitive that checks a number before it's asserted (evals, leaderboards, backtests), returns a signed reproducibility receipt, and meters itself (prepaid credits + x402 pay-per-call).
pypi MIT -
Camada semântica governada exposta a agentes de IA via MCP · o agente escolhe métricas de um vocabulário fechado e nunca escreve SQL · dbt + DuckDB + evals
MIT -
LLM observability and evals in a single binary: traces, prompts, datasets, scores and review queues for LLM and agent apps. Self-hosted, SQLite inside, OpenTelemetry-native, MCP and CLI for coding agents.
go Apache-2.0 ★ 1