MCP servers that ship as code — a repository or a package you install and run locally (usually over stdio). They have no hosted endpoint, so we cannot probe or grade them: instead each one shows its repository, install command, license, stars and when it was last updated.
Collected from GitHub repositories and npm, PyPI, Cargo, Go, Maven, NuGet and RubyGems packages tagged as MCP servers, the official MCP Registry, Docker's MCP catalog, ModelScope, Baidu MCP World, Cline and other public directories; one entry per repository. A server that also offers a hosted endpoint is listed under Remote instead. These entries are not counted in our reachability, verification or readiness statistics.
-
Model Context Protocol (MCP) server for intelligent task management, evaluation scoring, and session-based workflow tracking. Seamlessly integrates with AI assistants to provide structured task orchestration, real-time progress monitoring, and comprehensi
npm MIT -
An MCP server for the PGS Catalog REST API — the EMBL-EBI catalog of published polygenic scores, their traits, publications, evaluation samples, and reported performance metrics.
Apache-2.0 ★ 1 -
kryve-agent-eval-mcpgithub.com/louislaurent1/kryve-agent-eval-mcpupdated this year
Open-source MCP server for designing and scoring practical AI-agent evaluations.
MIT -
evalmcpgithub.com/gashel01/evalmcpupdated this year
Evaluation for AI agents — judge-based scoring and native RAG metrics (faithfulness, relevancy, context precision/recall). Python lib · CLI · MCP server · FastAPI.
Apache-2.0 ★ 1 -
promptup-plugingithub.com/gabikreal1/promptup-pluginupdated this year
AI coding skill evaluator for Claude Code — 11-dimension scoring, decision intelligence, PR reports
npm MIT -
binevalgithub.com/darksolitaire9-hub/binevalupdated this year
A deterministic, Rust-native binary evaluation kernel for AI safety, designed to help autonomous agents self-correct and red-team LLM infrastructure without fuzzy scores.
MIT -
MCP server for the PGS Catalog: published polygenic scores, the traits they predict, development/evaluation ancestry, per-score licences, performance metrics and source publications, via the official REST API.
MIT -
rag-eval-enginegithub.com/aptsalt/rag-eval-engineupdated this year
Production RAG with continuous evaluation — hybrid retrieval (BM25 + vector/RRF), LLM-as-judge scoring on faithfulness/relevance/hallucination, MCP server, adaptive retrieval tuning. 68 tests.
MIT -
metricmate-mcpgithub.com/marilynceo/metricmate-mcpupdated this year
LLM Output Quality Evaluation u2014 quantify, evaluate, and track LLM response quality: relevance, accuracy, tone, hallucination scoring, and consistency checks
-
Model Context Protocol (MCP) server for intelligent task management, evaluation scoring, and session-based workflow tracking. Seamlessly integrates with AI assistants to provide structured task orchestration, real-time progress monitoring, and comprehensive evaluation metrics.
npm MIT ★ 3 -
MCP server monorepo (FastMCP, Docker) — scored 100/100 agent-readiness in an external Factory Droid evaluation
MIT -
scern-mcpgithub.com/elcoryel/scern-mcpupdated this year
Product evaluation MCP server for US packaged food. Health scores, ingredient safety, regulatory flags, recall history, corporate ownership.
MIT ★ 1 -
Typed AI decisions for Codex, Claude and any MCP client, powered by TypeSafe Jev. Classify, score and evaluate with one generic tool.
MIT ★ 2 -
MCP server for discovering, evaluating, and comparing 17,000+ MCP servers from MCPpedia. Search by keyword, check security scores, compare alternatives, get install configs.
npm MIT ★ 2 -
MCP-native job search: deterministic sourcing + scoring, with Claude (the host model) as the job-fit evaluator. Inline review UI as an MCP App.
npm MIT -
🔌 Exposes AI Safety evaluation toolkit: care scoring, threat detection, burnout analysis, relationship prediction, and care membrane probe framework.
npm MIT -
🧭 Strategic BDI cognitive reasoning engine for AI agents. Goal decomposition DAGs, multi-factor utility scoring, risk evaluation, and adaptive replanning over MCP.
npm MIT ★ 13 -
Autonomous code-repair ReAct agent powered by FastMCP, AST pre-validation, token-efficient diff patching, and a scored 18-bug evaluation harness with live WebSocket streaming.
-
Strategic BDI reasoning engine for autonomous AI agents — goal decomposition, utility scoring, risk evaluation, and reactive replanning over PuterVision memory and world models.
npm MIT -
MCP server that carries what one session learned into the next. A per-project ledger written as things happen, scored per task by TypeSafe's Jev evaluation model via the Vercel AI Gateway.
npm MIT -
A comprehensive Model Context Protocol (MCP) server providing real-time YouTube Data API access for AI assistants. Features 14 functions including intelligent content evaluation with technology freshness scoring for knowledge base curation.
pypi MIT ★ 13 -
What your last session knew, scored against what this one is doing. MCP server: a per-project ledger written as things happen, recalled per task with TypeSafe's Jev evaluation model via Vercel AI Gateway.
MIT ★ 3 -
Synapse_OS is an agent-first project operations platform for managing multi-agent workflows, structured tasks, handovers, memory, and evaluator-based quality scoring. It combines FastAPI, Next.js, PostgreSQL/Qdrant, MCP/agent-tool APIs, and OpenAI/Playwright UAT testing.
MIT ★ 1 -
MO§ES™ — Enterprise AI Operator Evaluations. Marketing site + MCP server. mos2es.org
NOASSERTION -
Agent gateway for external tools — Rhumb Index scores services; Rhumb Resolve routes supported capability calls through governed execution rails.
npm MIT ★ 6 -
Model Context Protocol (MCP) server for multi-dimensional domain quality grading, radio test scoring, and trademark risk detection.
MIT -
brain-in-the-fishgithub.com/fabio-rovai/brain-in-the-fishupdated this year
Score any document. Prove every claim.
MIT ★ 101 -
Wasl-AI-github.com/krish2105/Wasl-AI-updated this year
Scores whether a business is legible to AI agents on a 100-point index, then generates the MCP server that makes it legible. Deterministic scoring, cited evidence, published refusals.
MIT -
Stop shipping agents on vibes. Score every agent output for quality, safety, and cost.
go MIT ★ 10 -
🛡️ Analyze transactions for fraud using advanced models and tools, providing insights and risk scores to enhance investigation efficiency.
Apache-2.0 ★ 1 -
Quorel is an MCP server that scores AI agent responses before they reach a user, a trust layer an agent calls mid-turn, not a dashboard someone checks after the fact.
NOASSERTION -
Function-calling quality gate for MCP/agent tools: scan tools and skills, lint description boundaries, and score model tool selection. Local-first CLI + MCP server.
MIT -
Regression testing for LLM applications in CI. Sample your reference and a score drop is read against how much that score moves on its own. A flip from pass to fail is always a regression. Python CLI, pytest plugin, MCP server.
pypi Apache-2.0 ★ 10 -
Touchstone — know if you broke your RAG. An MCP server and CLI that score retrieval quality against a golden set: recall@k, MRR and hit rate, with no LLM calls.
MIT ★ 1 -
MCP for MCPs — score artifacts against rubrics, explain failures and iterate until a defined quality threshold.
MIT -
FundScore — investor-readiness repo scorer. CLI + MCP server that evaluates GitHub repos for investor due diligence.
MIT -
Free MCP server: score any AI agent output 0-100 and get a PASS/PARTIAL/FAIL verdict (VOP verification). Verification only, no money moves.
MIT -
JPYC-commerce-mcpgithub.com/ackey-web/JPYC-commerce-mcpupdated this year
MCP server for AI agent commerce — task evaluation, trust-based negotiation, and JPYC payment execution on Polygon.
NOASSERTION -
hlido-mcpgithub.com/ankitkapur1992-hlido/hlido-mcpupdated this year
Hlido MCP server — independent AI-agent trust scores, claim audits & comparisons over JSON-RPC. Live at hlido.eu/mcp
MIT -
Detect pytest config-leakage that corrupts agent-benchmark grading. Deterministic, no-LLM. Catches the VulcanBench --cov addopts mis-scoring bug.
pypi MIT -
quorumgithub.com/marklynd/quorumupdated this year
A council of language models that scores a claim against a rubric, using only evidence you supply. Bounded timeouts, anonymised deliberation, hash-verified transcripts, MCP server.
MIT -
The-Gate-VPSgithub.com/EditorialOS/The-Gate-VPSupdated this year
Production deployment layer for The Gate. Multi-tenant MCP server exposing the eval engine as agent-callable tools — scored eval harness, draft→eval→revise agent example, PRD, runbook, Docker.
NOASSERTION -
poc_scrapergithub.com/pjnhek/poc_scraperupdated this year
Grounded GTM account research: ICP scoring and citation-traced outreach, exposed as a live hosted MCP server. Every claim traces to a numbered retrieval or gets dropped.
MIT -
Trust layer for AI agents. Credit-score-style ratings (300-850) for AI agent trustworthiness. MCP server for Claude, Cursor, and other MCP clients.
npm MIT ★ 1 -
Visible AI-driven robotics demos powered by VLM policies, MCP skills, and AI coding agents. Reviewable HTML reports — frames, maps, tool traces, scores — instead of buried terminal logs.
MIT ★ 7 -
LLM observability and evals in a single binary: traces, prompts, datasets, scores and review queues for LLM and agent apps. Self-hosted, SQLite inside, OpenTelemetry-native, MCP and CLI for coding agents.
go Apache-2.0 ★ 1 -
An agent that takes booking calls for a diagnostic clinic: finds the exam, holds a slot, and hands over to a person with a named reason. A hundred scored golden calls gate CI; a second reader asks a local model through Ollama, measured on the same calls; an MCP server exposes the clinic as tools. No model or API key needed to run it.
MIT -
Local-first SEC 8-K research lab: score filings point-in-time with Jev, OpenAI or local LLMs, and label which results can count as evidence. Never places orders.
MIT -
Glass Box Framework — runtime constitutional verification for AI answers. Trust Cards with claim-level reasoning chains, formal ECS scoring, the 7-angle Glassbox Court red team, and deterministic audit logs. MCP-native.
pypi Apache-2.0 ★ 10 -
Typed judgment tools for MCP agents. TypeSafe's Jev model as verify, screen, find, classify, rerank, decide, compare, extract, review, gate, and score: the model judges, policy decides auto, review, or escalate.
MIT ★ 49 -
Benchmark and MCP tools for LLM agents doing root-cause analysis on OpenTelemetry metrics, traces and logs. 20 recorded microservice incidents, every run scored against a 28-line rule and replayable. Only the strongest models clearly win.
Apache-2.0 -
Tool-using AI agent: finds AI/ML working student jobs in Berlin, scores them against a CV and drafts cover letters grounded in the CV. MCP server, hand-written agent loop, eval suite, human approval.
MIT -
Multi-agent LLM red-teaming and eval lab: attacks a RAG chatbot with prompt injection, poisoned documents and PII probes, scores replies via an MCP server (Presidio, DeepEval), and logs every run to SQLite for prompt and model comparison.
-
Provider-neutral prompt calibration engine with sensitivity-ranked meta-axes, hard-gate x soft-score fitness, walk-forward ship gates, guarded/expedition profiles, and explicit capability/risk reporting across Anthropic, OpenAI, Gemini, and local OpenAI-compatible backends. Optional sub-tools (mini-omega-lock, mini-antemortem-cli) distribute separately and plug in via the omegaprompt.preflight interface.
pypi Apache Software License