MCP servers that ship as code — a repository or a package you install and run locally (usually over stdio). They have no hosted endpoint, so we cannot probe or grade them: instead each one shows its repository, install command, license, stars and when it was last updated.
Collected from GitHub repositories and npm, PyPI, Cargo, Go, Maven, NuGet and RubyGems packages tagged as MCP servers, the official MCP Registry, Docker's MCP catalog, ModelScope, Baidu MCP World, Cline and other public directories; one entry per repository. A server that also offers a hosted endpoint is listed under Remote instead. These entries are not counted in our reachability, verification or readiness statistics.
-
MCP Server for AI-powered project innovation evaluation
-
Kaggle-Dataset-Evaluator-Agentgithub.com/VRYeshwanth/Kaggle-Dataset-Evaluator-Agentupdated this year
AI agent for dataset quality evaluation, ML task inference, and project recommendation using Google ADK and MCP.
-
Pluralistic human evaluation infrastructure for AI in production
pypi MIT ★ 2 -
FastMCP server that provides comprehensive security analysis for software packages across multiple ecosystems. It integrates seamlessly with Claude Desktop to provide AI-powered security evaluation capabilities.
MIT -
MO§ES™ — Enterprise AI Operator Evaluations. Marketing site + MCP server. mos2es.org
NOASSERTION -
A production-oriented enterprise AI agent with MCP tools, Agent Skills, tracing, reliability controls, and reproducible evaluations.
MIT -
evalmcpgithub.com/gashel01/evalmcpupdated this year
Evaluation for AI agents — judge-based scoring and native RAG metrics (faithfulness, relevancy, context precision/recall). Python lib · CLI · MCP server · FastAPI.
Apache-2.0 ★ 1 -
Self-hosted multi-agent AI platform: generates, evaluates and refines LLM answers until they hit a quality bar (~22% more accurate, ~45% fewer hallucinations)
MIT -
agent-eval-mcpgithub.com/nicolaemorcov/agent-eval-mcpupdated this year
Deterministic evaluation tools for AI coding agents, exposed as an MCP server.
pypi MIT -
ai-testing-mcpgithub.com/groovy-web/ai-testing-mcpupdated this year
MCP server for comprehensive AI testing, evaluation, and quality assurance
MIT ★ 19 -
botmark-mcpupdated this year
BotMark MCP Server — manage bot evaluations from Claude Desktop, Cursor, and other AI assistants
npm MIT -
kryve-agent-eval-mcpgithub.com/louislaurent1/kryve-agent-eval-mcpupdated this year
Open-source MCP server for designing and scoring practical AI-agent evaluations.
MIT -
agent-tank-mcpgithub.com/wassermanb01-petdetective/agent-tankupdated this year
MCP server for Agent Tank — submit agentic business pitches for AI shark evaluation
npm MIT -
Give your AI agents an evaluation tool - provider-agnostic LLM eval runner with an MCP server for Claude, structured output validation, datasets, evaluators and honest cost metrics
pypi MIT -
Coval MCP Server - AI evaluation platform integration for Claude Desktop, Cursor, and other MCP clients
npm MIT -
agentlensgithub.com/rrbanda/agentlensupdated this year
Evaluate, annotate, and gate AI agents you didn't build — conversational AgentOps on MLflow + MCP for OpenShift AI
Apache-2.0 -
@autousers/mcpgithub.com/autousers-ai/mcpupdated this year
Official MCP server for Autousers — UX evaluation, calibrated AI personas, side-by-side design review.
npm MIT -
AI-powered healthcare front-desk agent built with FastAPI, Groq, and MCP, featuring deterministic safety guardrails, patient-message triage, and an evaluation harness for messages.
-
Open source AI evaluation framework — LLM-as-judge + assertion-based evals for any AI app. CLI + MCP server.
npm Apache-2.0 -
HALO (Harness for Always-On Loop Oversight), from the survey Always-On Personal Intelligence: an evaluation harness and MCP plugin for always-on personal AI assistants and LLM agents. It checks whether Sensing, Memory, and Action are connected and whether self-evolving agents learn from outcomes, via paper and code audits and ablations.
MIT ★ 8 -
MCP server for Arize Phoenix — LLM tracing, evaluation, and dataset management via AI agents
MIT -
Production-shaped service-business AI agent with FastAPI, MCP, TTS, evaluation, Docker, and Kubernetes.
MIT -
vulca-visual-agent-plugingithub.com/vulca-org/vulca-visual-agent-pluginupdated this year
VULCA plugin for Claude Code & Cursor — 21 MCP tools + skills for AI cultural art creation & evaluation
NOASSERTION ★ 1 -
Time-travel debugging for AI agents: record, fork and replay runs, trajectory evaluation, MCP server
-
Permission-aware AI workflows for onboarding, IT access, vendor intake, and renewals. Python, MCP, Bedrock, durable approvals, and evaluation.
-
Enterprise-oriented AI agent platform with MCP tool integration, RAG, automated evaluation, regression detection, observability, and workflow automation.
MIT -
Applied AI reference application: agent workflows, RAG evaluation, MCP tools, durable approvals and crash recovery.
MIT -
Multi-agent AI marketing workflow platform (FastAPI, MCP Runner, React + TypeScript, Docker) with human-in-the-loop approval & offline evaluation harness
MIT -
Read-only MCP server that lets an AI assistant diagnose live trading bots, with an evaluation of 10 assistants
MIT -
AI assistant for equipment service teams: cited answers from product manuals, an approval-gated agent, MCP server, n8n workflow and evaluation.
MIT -
dox: tests how AI agents behave under adversarial inputs (duplicate refunds, prompt injection via tool output). Deterministic PASS/FAIL evaluator. Built on TrueForge + MCP.
-
agentguardgithub.com/glaborie/agentguardupdated this year
Production-grade QA layer for agentic AI - guardrails, RAG evaluation, MCP server, and drift monitoring in one self-hosted stack.
Apache-2.0 ★ 1 -
promptup-plugingithub.com/gabikreal1/promptup-pluginupdated this year
AI coding skill evaluator for Claude Code — 11-dimension scoring, decision intelligence, PR reports
npm MIT -
hunch_kitgithub.com/earlyprototype/hunch_kitupdated this year
Structured experimentation framework with human-in-the-loop evaluation, provider-agnostic execution, and MCP integration for AI-assisted experiment management.
MIT -
The experience graph for AI agents. Self-hosted memory, human-reviewed decisions, and outcome-backed datasets for model evaluation and learning.
npm Apache-2.0 ★ 23 -
governed-contextgithub.com/leonardoeverling-spec/governed-contextupdated this year
Governed memory for AI agents: explicit states, fail-closed sensitivity gates, quarantine-only writes (dependency-free MCP server) + a blind-evaluation protocol skill
MIT -
Personal, local-first conversation memory and evidence retrieval for AI coding assistants, with provenance, evaluation, and recovery.
MIT -
binevalgithub.com/darksolitaire9-hub/binevalupdated this year
A deterministic, Rust-native binary evaluation kernel for AI safety, designed to help autonomous agents self-correct and red-team LLM infrastructure without fuzzy scores.
MIT -
agent-eval-mcp-servergithub.com/mertdurukan/agent-eval-mcp-serverupdated this year
Production-grade MCP server that exposes AI agent quality evaluation (RAGAS faithfulness, precision, recall, response relevancy + custom JSON trace eval).
MIT -
EU AI Act governance layer for an enterprise application portfolio: Claude-based risk classifier, evaluation against a keyword baseline, and an MCP server with per-user scope.
MIT -
Persistent memory and context-engineering platform for AI agents with memory intelligence, hybrid retrieval, token-budgeted context compilation, custom LLM APIs, MCP, evaluation, observability, and a React control plane.
-
Cited-answer RAG service over 4 Czech labour laws: hybrid retrieval, LLM reranking, REST API (FastAPI) and MCP server for AI agents, offline evaluation with confidence intervals.
MIT -
@collapseindex/ci1t-mcpgithub.com/collapseindex/ci-1t-mcpupdated this year
MCP server for CI-1T — prediction stability engine. Evaluate model stability, detect ghosts, monitor fleet drift. 20 tools for AI agents.
npm SEE LICENSE IN LICENSE -
mcp-as-a-judgegithub.com/OtherVibes/mcp-as-a-judgeupdated this year
MCP as a Judge is a behavioral MCP that strengthens AI coding assistants by requiring explicit LLM evaluations
pypi MIT ★ 17 -
ML Referee evaluates whether candidate features add predictive value beyond a baseline for machine learning and forecasting. AI agents can preflight data, request quotes, and run authorized evaluations through MCP. Invite-only beta. Website: MCP endpoint: https://api.mlreferee.com/mcp
MIT -
Checks AI-written marketplace listings against the seller's item specifics before they're published: PUBLISH or REVIEW, with the exact words behind every unbacked claim. Deterministic, blind-evaluated, red-teamed (it broke; published).
MIT ★ 1 -
Two AI agents (a hand-written Claude tool loop and an ElevenAgents voice agent) sharing one read-only MCP tool server over synthetic billing data. Every figure is checked against tool results; known-answer evaluation, Docker, CI.
MIT -
Author, generate, validate and govern AI agent rules, skills, plugins and MCP configuration across 52 coding assistants. Review with verified model evidence, evaluate skills, lock and sign dependencies, and publish bundles. CLI, MCP server and Go API.
npm MIT ★ 171 -
An integrated AI platform composed from nine of these libraries: an authenticated multi-tenant API, OpenAI and Anthropic failover, streamed agent steps, an MCP server, per-tenant memory, durable multi-agent jobs with crash recovery, tracing, and evaluation.
MIT -
Log Claude Code sessions to Opik, the open-source LLM observability and evaluation platform, built by Comet. Tracing, evaluation, and skills for observable AI applications.
go Apache-2.0 ★ 25 -
Local-first, open-source platform for AI agents and business workflows with typed, calibrated decisions. Ready-to-run flows for finance, sales, IT and customer service; visual builder, human approvals, evaluations, RAG, MCP, REST API, SDK and CLI. One command on your own computer; Docker Compose for servers.
Apache-2.0 -
TraceMind-AIgithub.com/Mandark-droid/TraceMind-AIupdated this year
AI agent evaluation with MCP-powered intelligence
NOASSERTION ★ 1 -
Model Context Protocol (MCP) server for intelligent task management, evaluation scoring, and session-based workflow tracking. Seamlessly integrates with AI assistants to provide structured task orchestration, real-time progress monitoring, and comprehensive evaluation metrics.
npm MIT ★ 3 -
Bulwarkgithub.com/bpolania/bulwarkno update data
AI agent governance: content scanning, audit logs, policy evaluation, session management.
oci -
rca-ai-analystgithub.com/AndyH-78/rca-ai-analystupdated this year
Local AI tool to evaluate and improve Jira RCA quality (Streamlit + MCP + Ollama)
-
MCP server for evaluating Claude Code prompts with AI-powered analysis
★ 5 -
FairMindgithub.com/adhit-r/fairmindno update data
Release gate for AI systems from FairMind: evaluations, framework evidence and ledger snapshots.
pypi -
aimarket-mcp-servergithub.com/maxrobbins/aimarket-mcp-serverupdated this year
MCP Server for ai.market - Search, evaluate, and purchase datasets
pypi MIT -
A double-entry, multi-entity, tamper-evident ledger for AI agents (MCP server). Evaluation build.
mcpb NOASSERTION -
ai-safety-mcp-servergithub.com/Jack0319/ai-safety-mcp-serverupdated this year
A modular AI Safety MCP server providing safety evaluations, interpretability tools, and alignment-aware research utilities for agentic systems
Apache-2.0 ★ 1