Skip to content
Agentmart
Submit a tool

Iris

Stop shipping agents on vibes.

Connector by iris-eval · GitHub account verified by the MCP Registry

IndexedMITLocal (stdio)★ 9
Write a review

Chalk Labs

Need help deploying Iris?

Chalk Labs sets up agents like this inside real teams: connected to your tools, tested on your data, with a human in the loop where it matters.

Book a call

Install

Works with Claude Code, Codex, VS Code, Claude Desktop, Cursor, Windsurf. Read the repository before you give any tool access to your data.

npx agentmarket install connector/iris-eval-mcp-server

Asks which of your tools to add it to (Claude Code, Codex, Cursor, VS Code, Windsurf, Gemini CLI, Claude Desktop) and shows the change first. See https://agentmart.dev/cli.

Get the weekly diff

The weekly diff: one email a week with new agent tools, notable updates and grade changes, plus security alerts for tools you follow. Double opt-in, one-click unsubscribe.

We use your email only for this newsletter. Privacy

About

Stop shipping agents on vibes. Score every agent output for quality, safety, and cost.

Use cases: llm-evaluation, monitoring-observability

Reviews

Write a review

No reviews yet. Installed it? Reviews come from verified installers only. How reviews work

Permissions

What this server can do and what it asks for, from its tool list, install config and code. Checked Oct 25, 2026.

Tools
8 read · 3 write · 2 delete · 1 not yet classified
Secrets
5 (API keys, tokens or passwords)
Install scripts
None found
Runs as
Node.js package

Tools by what they do 14

Read from the source code; the running server may list different tools.

Read 8 · Look things up without changing anything.

  • compare_runs
  • compare_traces
  • evaluate_output
  • evaluate_runs
  • get_traces
  • list_rules
  • read_file
  • search

Write 3 · Create, change, send or run things.

  • deploy_rule
  • evaluate_with_llm_judge
  • verify_citations

Delete 2 · Delete, drop, revoke or wipe things.

  • delete_rule
  • delete_trace

Not yet classified 1 · We could not tell from the name and description.

  • log_trace

Secrets and settings it asks for

Secrets: values you paste in that let it act as you.

  • IRIS_ANTHROPIC_API_KEY
  • IRIS_API_KEY
  • IRIS_API_KEY_FILE
  • IRIS_OPENAI_API_KEY
  • IRIS_WEBHOOK_SECRET

Other settings (environment variables or headers).

  • IRIS_ALLOWED_ORIGINS
  • IRIS_ALLOW_UNAUTHENTICATED
  • IRIS_CITATION_ALLOW_FETCH
  • IRIS_CITATION_DOMAINS
  • IRIS_DASHBOARD
  • IRIS_DASHBOARD_HOST
  • IRIS_DASHBOARD_PORT
  • IRIS_DB_PATH
  • IRIS_HOME
  • IRIS_HOST
  • IRIS_LLM_JUDGE_MAX_COST_USD_PER_EVAL
  • IRIS_LOG_LEVEL
Show 14 more settings
  • IRIS_OTEL_ENDPOINT
  • IRIS_OTEL_HEADERS
  • IRIS_OTEL_SERVICE_NAME
  • IRIS_OTEL_TIMEOUT_MS
  • IRIS_PORT
  • IRIS_RELEVANCE_JUDGE_DAILY_BUDGET_USD
  • IRIS_RELEVANCE_JUDGE_MAX_CALLS_PER_REQUEST
  • IRIS_RELEVANCE_JUDGE_MODEL
  • IRIS_RELEVANCE_JUDGE_REDACT
  • IRIS_SEARCH_BUDGET_MS
  • IRIS_SEARCH_INDEX
  • IRIS_SQLITE_DRIVER
  • IRIS_TRANSPORT
  • IRIS_WEBHOOK_URL

Network hosts it talks to 60

  • api.github.com · found in code
  • api.test · found in code
  • arize.com · found in code
  • bit.ly · found in code
  • braintrust.dev · found in code
  • code.claude.com · found in code
  • code.visualstudio.com · found in code
  • collector.attacker.test · found in code
  • comet.com · found in code
  • company-todo-list.com · found in code
  • confident-ai.com · found in code
  • coreweave.com · found in code
  • cursor.com · found in code
  • deepeval.com · found in code
  • docs.cline.bot · found in code
  • docs.continue.dev · found in code
  • docs.coreweave.com · found in code
  • docs.devin.ai · found in code
  • docs.galileo.ai · found in code
  • docs.helicone.ai · found in code
  • docs.judgmentlabs.ai · found in code
  • docs.langchain.com · found in code
  • docs.latitude.so · found in code
  • docs.vendor.test · found in code
  • docs.wandb.ai · found in code
  • dora-website.com · found in code
  • drop.attacker.test · found in code
  • errors.pydantic.dev · found in code
  • eve-blog.com · found in code
  • facebook.com · found in code
Show 30 more hosts
  • galileo.ai · found in code
  • geminicli.com · found in code
  • ghcr.io · found in code
  • github.com · found in install command, code
  • glama.ai · found in code
  • helicone.ai · found in code
  • hiking-adventures.com · found in code
  • iana.org · found in code
  • informations.com · found in code
  • iris-eval.com · found in code
  • judgmentlabs.ai · found in code
  • langchain.com · found in code
  • langfuse.com · found in code
  • latitude.so · found in code
  • learn.chatgpt.com · found in code
  • mirror.attacker.test · found in code
  • my-website-234.com · found in code
  • our-company.com · found in code
  • password-reset-12bh12.com · found in code
  • patronus-ai.github.io · found in code
  • patronus.ai · found in code
  • promptfoo.dev · found in code
  • raw.githubusercontent.com · found in code
  • restaurant-zurich.com · found in code
  • sink.attacker.test · found in code
  • techservices.com · found in code
  • true-informations.com · found in code
  • vendor.test · found in code
  • wandb.ai · found in code
  • zed.dev · found in code

Files it touches

  • ~/.claude.json
  • ~/.ssh/id_ed25519
  • /etc/passwd
  • /home/iris/.iris
  • /tmp/iris-llm-smoke
  • %APPDATA%\\devin\\
  • Your home directory (~)
  • ~/.cline/data/settings/cline_mcp_settings.json
  • ~/.codex/config.toml
  • ~/.config/devin
  • ~/.continue/mcpServers/iris-eval.json
  • ~/.cursor/mcp.json
  • ~/.gemini/settings.json
  • ~/.iris
  • ~/.iris/iris.db
  • Any path the agent passes to its tools

Tools (14)

Read from the source code; the running server may differ. Descriptions are the server’s own words; hidden characters are shown as [U+…].

  • compare_runsCompare Runsread-only

    Description is built at runtime.

  • compare_tracesCompare Tracesread-only

    Description is built at runtime.

  • delete_ruleDelete or Disable Custom Rulecan change or delete data

    Description is built at runtime.

  • delete_traceDelete Tracecan change or delete data

    Description is built at runtime.

  • deploy_ruleDeploy Custom Rule

    Description is built at runtime.

  • evaluate_outputEvaluate Output

    Description is built at runtime.

  • evaluate_runsEvaluate Runs

    Description is built at runtime.

  • evaluate_with_llm_judgeEvaluate With LLM Judge

    Description is built at runtime.

  • get_tracesGet Tracesread-only

    Description is built at runtime.

  • list_rulesList Rulesread-only

    Description is built at runtime.

  • log_traceLog Trace

    Description is built at runtime.

  • read_fileread-only
  • searchread-only
  • verify_citationsVerify Citations

    Description is built at runtime.

Signals

Stars
9
Forks
4
Open issues
5
Weekly downloads
96
Last commit
Oct 22, 2026 (8 days ago)
Last release
Oct 25, 2026 (5 days ago)
Last package publish
Oct 4, 2026 (27 days ago)

From GitHub (fetched Oct 24, 2026) and the package registry.

Details

Registry name
io.github.iris-eval/mcp-server
Version
0.21.0
Auth
API key
Packages
npm @iris-eval/mcp-servermcpb https://github.com/iris-eval/mcp-server/releases/download/v0.21.0/iris-eval.mcpb
License
MIT
Repository
github.com/iris-eval/mcp-server
Website
iris-eval.com
First seen
Oct 24, 2026
Last checked
Oct 26, 2026
Source
Official MCP Registrygithub-graphql

Done for you

Rather not set it up yourself?

Chalk Labs scopes, builds and runs agents for your team: the task audit, the eval harness, the integrations and the weekly ops report. Fixed scope, priced before we start.

See how we build AI agents →Read the case studies

Chalk Labs is a two-founder AI and growth studio. We build AI agents, CRMs and MVPs for Web3 and AI companies, and have shipped 50+ product and token launches.

launches shipped
50+
launches shipped
media placements
500+
media placements
founders on every account
2
founders on every account

Press we've placed: Forbes · CoinDesk · Decrypt · Cointelegraph · The Block

Rahil Jain, Co-founder, Chalk Labs. Ex-Goldman Sachs engineer. You talk to the person who builds it.

Get Iris running in your business

Tell us what you want to automate. A founder replies within one working day with a plan, not a pitch deck.

Goes to the Chalk Labs founders, not the newsletter. Privacy

Need help deploying Iris?

CallGet help
Cite this page

Iris MCP server. Agent Mart. https://agentmart.dev/connectors/iris-eval-mcp-server (last updated 2026-10-26)