Oliver Backend Spec

Status: Go MVP (multi-tenant workspaces) · Updated: 2026-07-03

This backend module implements Oliver. Finished local-agent turns, connector records become immutable Postgres events. Context is derived on read from those raw events.

One rule governs the system: capture raw and immutable, derive the rest, resolve at read.

Architecture

write:  pipeline -> POST /events -> Go API -> Postgres
        agent hook listener -> POST /events -> Go API -> Postgres
        agent hook collector -> local queue -> POST /events -> Go API -> Postgres
                                                        -> deterministic classify -> Postgres
read:   client / agent -> GET /topics/{key} -> Go API -> Postgres -> digest summary
        client / agent -> POST /ask -> Go API -> extract topics -> Postgres -> digest summary

Module Layout

backend/
├── cmd/
│   ├── api/
│   ├── db-setup/
│   └── agent-hook-listener/
├── internal/
│   ├── contextlayer/
│   │   ├── auth/
│   │   ├── domain/
│   │   ├── http/
│   │   ├── rules/
│   │   ├── store/
│   │   └── summary/
│   └── ingestion/
├── ingestion/
├── config/
└── database/

frontend/
└── portal/

extensions/
├── agent-hooks/
└── chrome-extension/

infra/
├── aws/
├── docker/
├── local/

docs/
├── app/
├── internal/
└── public/
scripts/
pipelines/

frontend/ remains top-level because it contains frontend/client code. extensions/ contains installable local clients such as the Claude/Codex hook collector and Chrome extension. The backend module contains Go service code, the compatibility hook listener, schema, and config. Repo-root pipelines/, infra/, scripts/, and docs/ hold batch jobs, operational assets, and long-form internal and public docs.

API Contract

Machine-readable route and schema details live in backend/openapi/openapi.yaml.

The API is multi-tenant. Every scoped request authenticates a user with a bearer token (Authorization: Bearer usr_...) and selects a workspace with an X-Workspace: <slug> header. A user may read/write only in workspaces they belong to; a memberships row is the permission. actor is always the authenticated user; any client-provided actor is ignored. Postgres Row-Level Security isolates workspaces, so the API must connect as the non-BYPASSRLS context_app role from backend/database/roles.sql.

Identity And Workspaces

• Browser users sign up through Supabase; the API creates a local shadow user after verifying a Supabase JWT.
• POST /api-keys creates a user-scoped API key for the authenticated user. The token is shown once; only its sha256 hash is stored.
• POST /workspaces creates a workspace for the authenticated user, who becomes owner.
• GET /workspaces lists the caller's workspaces and roles.
• GET /workspaces/{slug}/members lists workspace members.
• POST /workspaces/{slug}/members lets an owner grant an existing user access by email.
• cmd/issue-token mints tokens for non-interactive identities such as agents and connectors, optionally joining them to a workspace.

POST /events

{
  "ts": "2026-07-01T15:42:10Z",
  "session_id": "s1",
  "kind": "agent_task",
  "intent": "the raw prompt or source title",
  "output": "the raw final message or source body",
  "targets": [{ "type": "path", "value": "internal/contextlayer/store/store.go" }],
  "idempotency_key": "stable-client-retry-key",
  "model": "optional-model-name",
  "usage": { "input_tokens": 0, "output_tokens": 0 }
}

Behavior:

1. Generate a ULID-like event_id.
2. Attribute actor to the authenticated user; the workspace is taken from current_setting('app.workspace_id'), never the request body.
3. Compute intent_key from normalized intent for grouping, not deduplication.
4. If idempotency_key conflicts within the workspace, return the existing event id.
5. Insert the event, upsert its artifacts, and record touched verb-edge actions in one transaction.
6. Classify deterministic tags in a background goroutine.

GET /topics and GET /topics/{key}

Topics are the addressable unit of context. A topic key is namespaced, for example component:storage.

GET /topics lists the topics (key, title, description, event_count, last_activity), most active first.

GET /topics/{key} returns one topic's summary. Behavior:

1. Resolve members from matching manual/newest automated classification rows plus current path rules.
2. Check the disposable summaries cache against the member high-water mark.
3. On a miss, take a Postgres advisory lock, re-check, generate a digest summary from the most recent SUMMARY_MAX_EVENTS, and upsert the cache row.
4. Return { key, title, body, source_event_ids, provenance }.

POST /ask

{ "query": "What's the status of the backend?" }

A key-agnostic question: the caller does not name a topic; the backend extracts which topic(s) the question is about and returns their data. Behavior:

1. Require a non-empty query.
2. Tokenize the query (dropping question/status stopwords) and score every topic by how many terms overlap its key value, title, and description.
3. Resolve the top matches (up to 3) exactly like GET /topics/{key}.
4. Return { query, matched_keys, topics: [ { key, title, body, source_event_ids, provenance } ] }. No match yields an empty list, not an error.

Topic extraction is deterministic — no LLM provider is required, so /ask works in every environment.

Other Endpoints

• GET /events lists recent events, optionally windowed by timestamp, duration, or since_hours.
• GET /files lists connector documents touched by events.
• GET /health returns { "status": "ok" }.

Data Rules

• Every tenant-owned table carries workspace_id and is isolated by Row-Level Security, scoped to the app.workspace_id / app.user_id set per request. workspaces, users, and api_tokens are global because they are resolved before a workspace is selected.
• Authentication is user-scoped (api_tokens.token_hash maps to user_id); authorization is a memberships(workspace_id, user_id, role) row. Tokens are never workspace-bound.
• Source tables events, actions, and classifications are append-only. Database triggers reject update and delete.
• artifacts and topics are mutable identity tables (upserted on write). summaries is the disposable cache.
• Path targets must be repo-relative POSIX paths.
• Every topic key must be namespaced: component:storage, project:oliver, topic:caching.
• Manual classifications are always preserved. Among automated classifier versions, only the newest version for an event participates in resolved tags.

Ingestion

Finite fetch sources live in repo-root Python pipelines:

• notion
• google_docs
• slack
• github

Each pipeline requires the Oliver API to be running and must post normalized POST /events payloads with Authorization: Bearer ... and X-Workspace headers, stable targets, and idempotency keys.

extensions/agent-hooks is the default local path for Claude/Codex developer-agent events. It installs repo-local hook config and stores native hook payloads under .oliver/agent-hooks. Its uploader normalizes queued payloads into local_agent_event records and posts them to authenticated POST /events.

cmd/agent-hook-listener is a localhost adapter for direct HTTP hooks on /agent-hooks/{agent}; it normalizes each payload and posts the resulting event to POST /events.

Acceptance Tests

Run from backend/:

go test ./...

The suite covers health, verbatim event storage, authenticated actor attribution, workspace membership isolation, idempotency, deterministic tags, topic listing and summaries, key-agnostic /ask topic extraction, cache invalidation, append-only triggers, and concurrent inserts.

Phase Next

• Friendlier workspace invites for emails that have not signed up yet, such as a claim code or pending-invite flow.
• Per-workspace summary-cache invalidation on path-rule change.
• Switch the deployed runtime DATABASE_URL to the context_app role so RLS isolates.
• Native Go Anthropic classifier and summary integrations.
• Provider-agnostic LLM client configuration beyond the first Anthropic integration.
• Native Go S3/R2 output offload and bounded hydration.
• Durable classifier queue/backfill.
• Outcome watchers for merge/revert/supersede signals.
• Denormalized resolved tags for larger installations.