OpenAI Evals
Framework and registry of evals for testing LLMs and LLM-based systems, maintained by OpenAI
Observability tool by OpenAI
Chalk Labs
Need help deploying OpenAI Evals?
Chalk Labs sets up agents like this inside real teams: connected to your tools, tested on your data, with a human in the loop where it matters.
Get started
Setup steps are in the repository. Website · Docs · Repository
About
Use cases: llm-evaluation
Reviews
Write a reviewNo reviews yet. Installed it? Reviews come from verified installers only. How reviews work
Fact sheet
- Category
- Evals · open source
Facts read from the vendor's own pages by claude-haiku-5-5 on Oct 24, 2026; a fact is shown only with the page it came from.
Signals
- Stars
- 19,644
- Forks
- 3,120
- Open issues
- 180
- Last commit
- Apr 14, 2026 (6 months ago)
From GitHub (fetched Oct 24, 2026).
Details
- License
- License unknown
- Repository
- github.com/openai/evals
- Website
- github.com/openai/evals
- First seen
- Oct 24, 2026
- Last checked
- Oct 24, 2026
- Source
- READMEDocsgithub-graphql
More from OpenAI
- 0.708score
security-review-openai
OpenAI
Perform language and framework specific security best-practice reviews and suggest improvements.
SkillBFreshness: B★ 895Claude Code +9 - 0.708score
xlsx-processing-openai
OpenAI
Toolkit for comprehensive Spreadsheet reading, creation, editing, and analysis with visual quality control.
SkillBFreshness: B★ 895Claude Code +9 - 0.708score
docx-processing-openai
OpenAI
Toolkit for comprehensive document reading, editing, and creation with visual quality control.
SkillBFreshness: B★ 895Claude Code +9 - 0.704score
skill-creator-openai
OpenAI
Guide for creating effective skills.
SkillBFreshness: B★ 895Claude Code +9 - 0.703score
pdf-processing-openai
OpenAI
Toolkit for comprehensive PDF reading, reviwing, and creation with visual quality control.
SkillBFreshness: B★ 895Claude Code +9 - 0.695score
OpenAI Codex
OpenAI
OpenAI's coding agent for the terminal, editor, desktop app and ChatGPT cloud, included in ChatGPT plans
AgentAFreshness: A★ 130.8K↓ 13.1M/wk
Done for you
Rather not set it up yourself?
Chalk Labs scopes, builds and runs agents for your team: the task audit, the eval harness, the integrations and the weekly ops report. Fixed scope, priced before we start.
See how we build AI agents →Read the case studies
Chalk Labs is a two-founder AI and growth studio. We build AI agents, CRMs and MVPs for Web3 and AI companies, and have shipped 50+ product and token launches.
- launches shipped
- 50+
- launches shipped
- media placements
- 500+
- media placements
- founders on every account
- 2
- founders on every account
Press we've placed: Forbes · CoinDesk · Decrypt · Cointelegraph · The Block
Rahil Jain, Co-founder, Chalk Labs. Ex-Goldman Sachs engineer. You talk to the person who builds it.
Get OpenAI Evals running in your business
Tell us what you want to automate. A founder replies within one working day with a plan, not a pitch deck.
Cite this page
OpenAI Evals. Agent Mart. https://agentmart.dev/observability/openai-evals (last updated 2026-10-24)