MCP servers that ship as code — a repository or a package you install and run locally (usually over stdio). They have no hosted endpoint, so we cannot probe or grade them: instead each one shows its repository, install command, license, stars and when it was last updated.
Collected from GitHub repositories and npm, PyPI, Cargo, Go, Maven, NuGet and RubyGems packages tagged as MCP servers, the official MCP Registry, Docker's MCP catalog, ModelScope, Baidu MCP World, Cline and other public directories; one entry per repository. A server that also offers a hosted endpoint is listed under Remote instead. These entries are not counted in our reachability, verification or readiness statistics.
-
llm-chess-evaluation-harnessgithub.com/PythonicVarun/llm-chess-evaluation-harnessupdated this year
LLM Chess Evaluation Harness ♟️
MIT -
FastMCP server that provides comprehensive security analysis for software packages across multiple ecosystems. It integrates seamlessly with Claude Desktop to provide AI-powered security evaluation capabilities.
MIT -
GenPark Reasoning MCTS Step Verity Evaluator - Implements Monte Carlo Tree Search (MCTS) Step-Level Process Reward Modeling (PRM) & Hallucination Branch Pruning.
MIT -
Pluralistic human evaluation infrastructure for AI in production
MIT ★ 2 -
MO§ES™ — Enterprise AI Operator Evaluations. Marketing site + MCP server. mos2es.org
NOASSERTION -
Self-hosted multi-agent AI platform: generates, evaluates and refines LLM answers until they hit a quality bar (~22% more accurate, ~45% fewer hallucinations)
MIT -
Give your AI agents an evaluation tool - provider-agnostic LLM eval runner with an MCP server for Claude, structured output validation, datasets, evaluators and honest cost metrics
MIT -
MCP Server for xCOMET translation quality evaluation
npm MIT -
HALO (Harness for Always-On Loop Oversight), from the survey Always-On Personal Intelligence: an evaluation harness and MCP plugin for always-on personal AI assistants and LLM agents. It checks whether Sensing, Memory, and Action are connected and whether self-evolving agents learn from outcomes, via paper and code audits and ablations.
MIT ★ 8 -
MCP server for mathematical expression evaluation with strict grammar validation
npm MIT -
Standalone static preflight checks for LLM evaluation configurations, leakage detection and canonical risk rules.
Apache-2.0 ★ 1 -
Standalone measurement-reliability probes and empirical noise-ceiling estimation for repeated LLM evaluations.
Apache-2.0 ★ 1 -
ML Referee evaluates whether candidate features add predictive value beyond a baseline for machine learning and forecasting. AI agents can preflight data, request quotes, and run authorized evaluations through MCP. Invite-only beta. Website: MCP endpoint: https://api.mlreferee.com/mcp
MIT -
promptup-plugingithub.com/gabikreal1/promptup-pluginupdated this year
AI coding skill evaluator for Claude Code — 11-dimension scoring, decision intelligence, PR reports
npm MIT -
governed-contextgithub.com/leonardoeverling-spec/governed-contextupdated this year
Governed memory for AI agents: explicit states, fail-closed sensitivity gates, quarantine-only writes (dependency-free MCP server) + a blind-evaluation protocol skill
MIT -
binevalgithub.com/darksolitaire9-hub/binevalupdated this year
A deterministic, Rust-native binary evaluation kernel for AI safety, designed to help autonomous agents self-correct and red-team LLM infrastructure without fuzzy scores.
MIT -
The experience graph for AI agents. Self-hosted memory, human-reviewed decisions, and outcome-backed datasets for model evaluation and learning.
Apache-2.0 ★ 23 -
Local-first MCP server for versioning, searching, evaluating, and improving AI prompts with coding agents.
npm MIT -
MCP Server implementation of AgentFold - Long-Horizon Web Agents with Proactive Context Management. v2.0 adds analytics, validation, quality evaluation, and performance benchmarking.
npm MIT -
Self-hosted data annotation and labeling platform for images, video, audio, text, PDF and LLM evaluation. Your storage, your models, your cloud.
NOASSERTION -
A Model Context Protocol server with machine learning engineering tools: BPE token counting and cost estimates, BM25 search, paired bootstrap and McNemar evaluation comparison, JSON Schema validation, and prompt-injection and PII scanning.
MIT -
Checks AI-written marketplace listings against the seller's item specifics before they're published: PUBLISH or REVIEW, with the exact words behind every unbacked claim. Deterministic, blind-evaluated, red-teamed (it broke; published).
MIT ★ 1 -
Universal Model Context Protocol (MCP) server and CLI runner for TypeSafe Jev decision models. Calibrated evaluation for code reviews, acceptance gating, and agent guardrails across Claude Code, OMP, and Cursor.
MIT ★ 2 -
An integrated AI platform composed from nine of these libraries: an authenticated multi-tenant API, OpenAI and Anthropic failover, streamed agent steps, an MCP server, per-tenant memory, durable multi-agent jobs with crash recovery, tracing, and evaluation.
MIT -
Model Context Protocol (MCP) server for intelligent task management, evaluation scoring, and session-based workflow tracking. Seamlessly integrates with AI assistants to provide structured task orchestration, real-time progress monitoring, and comprehensi
npm MIT -
metricmate-mcpgithub.com/marilynceo/metricmate-mcpupdated this year
LLM Output Quality Evaluation u2014 quantify, evaluate, and track LLM response quality: relevance, accuracy, tone, hallucination scoring, and consistency checks
-
Log Claude Code sessions to Opik, the open-source LLM observability and evaluation platform, built by Comet. Tracing, evaluation, and skills for observable AI applications.
Apache-2.0 ★ 25 -
Model Context Protocol (MCP) server for intelligent task management, evaluation scoring, and session-based workflow tracking. Seamlessly integrates with AI assistants to provide structured task orchestration, real-time progress monitoring, and comprehensive evaluation metrics.
MIT ★ 3 -
Translation quality evaluation MCP Server powered by xCOMET (eXplainable COMET).
MIT ★ 2 -
Omneum: Agent Context Reliabilityno update data
Evaluate source reliability, dependent evidence, and conflicting claims in your agent's context.
smithery -
mathexpr-mcpgithub.com/MukundaKatta/mathexpr-mcpupdated this year
MCP server: evaluate and simplify mathematical expressions (mathjs).
MIT -
Reduce the time it takes to evaluate SRG/STIG compliance.
MIT -
MCP server for Recruitee service for easy LLM evaluation
★ 3 -
MCP Server for Reftrix - AI agent integration for web design analysis, layout extraction, motion detection, and quality evaluation
npm AGPL-3.0-only -
SLAC-MCPgithub.com/DennisPrediger/SLAC-MCPupdated this year
MCP Server to expose evaluation Tools to LLM and AI Agents using a Simple Logic & Arithmetic Compiler
Apache-2.0 -
MCP server for Nia AI WebEvalAgent - Web automation and evaluation agent
npm MIT -
A secure Model Context Protocol (MCP) server for mathematical expression evaluation with strict grammar validation.
-
Eval_MCPgithub.com/AyushSaha184/Eval_MCPupdated this year
An MCP server that offers 15 specialized tools to evaluate and improve the accuracy of RAG based systems.
★ 1 -
A TypeScript implementation of a Model Context Protocol (MCP) server, designed to evaluate MCP capabilities over/with RAG (Retrieval-Augmented Generation)
★ 1 -
pipeworx-mcp-mathjsupdated this year
MCP server for mathematical expression evaluation and unit conversion via mathjs API
npm MIT -
MCP server for discovering, evaluating, and comparing MCP servers via the MCPpedia catalog
npm MIT -
agent-negotiation-mcpgithub.com/csoai-org/agent-negotiation-mcpupdated this year
AI-powered agent negotiation MCP server for agents. Supports propose deal, evaluate offer, counter offer. By MEOK AI Labs.
pypi MIT License -
Universal code + logic calculator for AI models: execute 31 languages, evaluate logic, analyze complexity. Exposed via MCP.
pypi Apache-2.0 -
Turn conda-forge, Spack, and EasyBuild recipes into Resolvo-solved variants, CycloneDX SBOMs, and build-evaluated package bundles
MIT ★ 1 -
fossick-mcpgithub.com/lipdog/fossick-mcpupdated this year
MCP server — search code, find libraries, and evaluate dependencies across all of GitHub, PyPI, and npm
pypi MIT -
A high-fidelity, local-first MCP server and protocol for storing, versioning, syncing, and evaluating AI agent prompts and skills
MIT ★ 3 -
mcp-prompt-labgithub.com/nikolasbarwicki/mcp-prompt-labupdated this year
Prompt evaluation workbench as an MCP server — run evals, compare providers, and test assertions directly from Claude Code or any MCP host.
MIT -
Simple Rust MCP server providing the current time and mathematical expression evaluation over SSE.
-
NIA AI Web Evaluation Agent MCP Server - Autonomous browser testing and debugging
npm MIT -
Modular, observable RAG framework exposing tools via MCP (Model Context Protocol) — hybrid search, multimodal ingestion, evaluation, dashboard
MIT -
MCP server for Formulon Excel-compatible formula and workbook evaluation
npm Apache-2.0 -
scern-mcpgithub.com/elcoryel/scern-mcpupdated this year
Product evaluation MCP server for US packaged food. Health scores, ingredient safety, regulatory flags, recall history, corporate ownership.
MIT ★ 1 -
Typed AI decisions for Codex, Claude and any MCP client, powered by TypeSafe Jev. Classify, score and evaluate with one generic tool.
MIT ★ 2 -
Beta x402 text inspection with bounded decoding, canary tripwires and signed inspection receipts. Explicit evaluation limits; 0.003 Base USDC.
MIT -
hive-market-mcpgithub.com/atomicintuition/hive_marketplaceupdated this year
Hive Market MCP Server — discover, evaluate, and install MCP tools for AI agents
npm MIT -
AI Guardian that challenges actions before execution. Forces clarity of thought through adversarial questioning and NLI-based reasoning evaluation.
-
echoloc-mcpgithub.com/echoloc-ai/echoloc-mcpupdated this year
Remote MCP server for echoloc company technographics — search 760K+ companies by tech stack with direction of change (adopting/replacing/evaluating)
MIT -
path402github.com/b0ase/path402updated this year
The $402 Protocol — MCP server for AI agents to discover, evaluate, and acquire tokenized content
npm MIT -
@kevinrabun/judgesgithub.com/kevinrabun/judgesupdated this year
45 specialized judges that evaluate AI-generated code for security, cost, and quality.
npm MIT -
regex-toolkit-mcpgithub.com/vinkius-labs/regex-toolkit-mcpupdated this year
Vinkius Edge high-performance Model Context Protocol (MCP) server for evaluating and testing regular expressions.
★ 3