AGENTS.md · topoteretes/cognee

Sections

1. Repository Guidelines
2. Project Structure & Module Organization
3. Build, Test, and Development Commands
4. Runtime Flags Worth Knowing
5. Multi-Tenancy Support by Backend
6. Coding Style & Naming Conventions
7. Testing Guidelines
8. Commit & Pull Request Guidelines
9. CI Mirrors Local Commands

First lines (shown under the repo's Apache-2.0 license)

## Repository Guidelines

This document summarizes how to work with the cognee repository: how it’s organized, how to build, test, lint, and contribute. It mirrors our actual tooling and CI while providing quick commands for local development.

## Design Principles

1. **Don't overengineer**: Simple beats complex.
2. **No fallbacks**: One correct path, no alternatives.
3. **One way**: One way to do things, not many.
4. **Throw errors**: Fail fast when the operation's preconditions aren't met. Observability (telemetry, logging) is best-effort and never changes the outcome of the operation.
5. **No backups**: Trust the primary mechanism.
6. **Separation of concerns**: Each function has a single responsibility.
7. **Be concise but informative**: No essays, but the docstrings and comments a reader needs.
8. **No invented limits**: Never add a cutoff (`max_n`, `[:k]`, "older than X") or an expiry (TTL, cooldown, eviction) the task did not ask for — both are silent data loss. If you believe one is unavoidable, stop and ask, stating what would be dropped and why; never add it first and justify it afterwards. Existing bounds (lock leases, recovery floors, batch caps) are deliberate: removing or changing one is the same question.

**Methodology**: surgical changes only; evidence-based debugging (minimal, targeted logging); fix root causes, not symptoms. Debug like a detective: form the theory of the crime, collect evidence, and change code only once the evidence proves it.

## Project Structure & Module Organization

- `cognee/`: Core Python library and API.
  - `api/`: FastAPI application and versioned routers under `api/v1/` — memory API (`remember`, `recall`, `improve`, `forget`), low-level operations (`add`, `cognify`, `search`, `delete`, `update`), and supporting routers (`datasets`, `users`, `permissions`, `settings`, `sync`, `visualize`, `health`, `skills`, `sessions`, …). Each router package has a `routers/` folder; `api/client.py` registers them.
  - `cli/`: CLI entry points and subcommands invoked via `cognee` / `cognee-cli`.
  - `infrastructure/`: Databases, LLM providers, embeddings, loaders, and storage adapters.
  - `modules/`: Domain logic (graph, retrieval, ontology, users, processing, observability, etc.).
  - `tasks/`: Reusable tasks (e.g., code graph, web scraping, storage). Extend with new tasks here.
  - `eval_framework/`: Evaluation utilities and adapters.
  - `shared/`: Cross-cutting helpers (logging, settings, utils).
  - `tests/`: Unit, integration, CLI, and end-to-end tests organized by feature.
  - `__main__.py`: Entrypoint to route to CLI.
- `cognee-mcp/`: Model Context Protocol server exposing cognee as MCP tools (SSE/HTTP/stdio). Contains its own README and Dockerfile.
- `cognee-frontend/`: Next.js UI for local development and demos.
- `distributed/deploy/`: One-click deployment templates (Modal, Fly.io, Railway, Render, Daytona).
- `examples/`: Example scripts demonstrating the public APIs and features (graph, code graph, multimodal, permissions, etc.).
- `notebooks/`: Jupyter notebooks for demos and tutorials.
- `.agents/skills/`: Task-specific agent skills. This is the only copy; add and edit skills here (Codex reads this folder directly).
- `.claude/skills/`: Symlink to `../.agents/skills` so Claude Code finds the same skills. On Windows clones without symlink support, the SessionStart hook in `.claude/settings.json` (`.claude/hooks/link-skills.sh`) replaces it with a directory junction automatically; no manual step is needed. That hook marks the path skip-worktree, so on those clones only, a later change to the `.claude/skills` entry itself makes git refuse to update it — run `git update-index --no-skip-worktree .claude/skills && git checkout -- .claude/skills`, then start a new session.
- `cognee/alembic/`: Relational (Alembic) schema migrations; `alembic.ini` is in `cognee/`. Graph/vector data migrations live in `cognee/modules/migrations/`.

Notes:
- Co-locate feature-specific helpers under their respective package (`modules/`, `infrastructure/`, or `tasks/`).
- Extend the system by adding new tasks, loaders, or retrievers rather than modifying core pipeline mechanisms.

## Build, Test, and Development Commands

Python (root) – requires Python >= 3.10 and < 3.15 (i.e. 3.10–3.14, see `pyproject.toml`). We recommend `uv` for speed and reproducibility.

- Create/refresh env and install dev deps:
```bash
uv sync --dev --all-extras --reinstall
```

- Run the CLI (examples). The memory commands are the primary surface; `add`/`cognify`/`search` are the low-level stages they call underneath:
```bash
uv run cognee-cli remember "Cognee turns documents into AI memory."   # add + cognify (+ improve)
uv run cognee-cli recall "What does cognee do?"
uv run cognee-cli improve -d main_dataset   # enrich / index the graph
uv run cognee-cli forget --all              # NOTE: no confirmation prompt
uv run cognee-cli add "..." && uv run cognee-cli cognify && uv run cognee-cli search "..."   # low level
uv run cognee-cli -ui   # Launches UI, backend API, and MCP server together
```

- Start the FastAPI server directly:
```bash
uv run python -m cognee.api.client
```

- Run tests (CI mirrors these commands):
```bash
uv run pytest cognee/tests/unit/ -v
uv run pytest cognee/tests/integration/ -v
```

- Lint and format (ruff):
```bash
uv run ruff check .
uv run ruff format .
```

- Optional static type checks (ty):
```bash
uv run ty check .
```

MCP Server (`cognee-mcp/`):

- Install and run locally:
```bash
cd cognee-mcp
uv sync --dev --all-extras --reinstall
uv run python src/server.py               # stdio (default)
uv run python src/server.py --transport sse
uv run python src/server.py --transport http --host 127.0.0.1 --port 8000 --path /mcp
```

- API Mode (connect to a running Cognee API):
```bash
uv run python src/server.py --transport sse --api-url http://localhost:8000 --api-token YOUR_TOKEN
```

- Docker quickstart (examples): see `cognee-mcp/README.md` for full details
```bash
docker run -e TRANSPORT_MODE=http --env-file ./.env -p 8000:8000 --rm -it cognee/cognee-mcp:main
```

Frontend (`cognee-frontend/`):
```bash
cd cognee-frontend
npm install
npm run dev     # Next.js dev server
npm run lint    # ESLint
npm run build && npm start
```

## Runtime Flags Worth Knowing

Three env flags trade memory features for speed; know what each disables before flipping it:

- `CACHING` (default `true`) — master switch for the session-memory layer. When `false`,
  `remember(session_id=...)` raises, `recall()` loses session history, `agent_memory`
  session options error out, and `AUTO_FEEDBACK` is implicitly disabled. Never benchmark
  cognee with this off — that measures cognee with its memory layer removed.
- `AUTO_FEEDBACK` (default `true`) — one structured-output LLM call per answered turn
  that detects implicit feedback and lets memory self-tune. Disable for low-latency,
  lower-cost reads; session store/recall itself keeps working.
- `DATASET_QUEUE_ENABLED` (default `true`) — per-process cap on concurrent datasets
  (`DATASET_QUEUE_MAX_CONCURRENT`, default 6); also tears down subprocess DB engines on
  scope exit and pins in-use engines against cache eviction. Only engages when
  `ENABLE_BACKEND_ACCESS_CONTROL` is on (its default) — with access control off the
  flag is a no-op, so flipping it cannot affect performance there. Disable only for
  single-dataset scripts — under parallel multi-dataset load, turning it off risks
  file-lock leaks and unbounded embedded engines.

## Multi-Tenancy Support by Backend

With `ENABLE_BACKEND_ACCESS_CONTROL=true` (the default) each user+dataset gets isolated
graph and vector databases. Backend support (source of truth:
`cognee/infrastructure/databases/dataset_database_handler/supported_dataset_database_handlers.py`):

- Graph — supported: Ladybug/Kuzu (default), Neo4j (needs multi-database, i.e.
  Enterprise/Aura), Postgres (demo), Turso. Unsupported: Neptune, ladybug-remote.
- Vector — supported: LanceDB (default), PGVector, Turso. Unsupported: Neptune
  Analytics and community adapters (unless they register a handler via
  `use_dataset_database_handler()`).
- Relational (SQLite/Postgres/Turso) is always a single shared DB (users, ACLs, registry).
- Turso: the three layers and the session cache (`CACHE_BACKEND=turso`) can run on the Turso
  rewrite engine (`pyturso`, local files only);
  setup, journal modes and engine limits are in `docs/turso-local.md`.

Each provider has a default handler derived from it, so no handler setting is needed
normally. Setting `GRAPH_DATASET_DATABASE_HANDLER` / `VECTOR_DATASET_DATABASE_HANDLER`
explicitly is never overwritten by that derivation, and is how you select the alternates:
`neo4j_community` (one Docker container per dataset, for Community edition, which serves
one database per server), `neo4j_aura_dev` (one Aura instance per dataset; dev/PoC only),
`postgres_graph_shared` and `pgvector_shared` (a schema per dataset in cognee's main
Postgres database, so no `CREATE DATABASE` privilege is needed). Each requires its
matching provider. See CLAUDE.md for the full matrix.

Both graph and vector must be supported, or cognee raises `EnvironmentError` — an
unsupported backend with the flag on is a hard error, not a fallback to shared DBs;
set `ENABLE_BACKEND_ACCESS_CONTROL=false` to run such backends single-tenant.