Architecture.

Nine deployable components, one canonical store, and a seam between the TypeScript and Python halves that is built but not yet carrying traffic.

Services

CLIENTSSIEM / SOAR / EDRBearer keyPortalNext.jsSDKsts · pythonEDGEAPI gatewayHono · REST + TAXIIWORKERS + ENGINEPostgres + pgvectorcanonical storeIngestion workers10 connectorsIntel enginepython · behind seamown storedashed = writes · the engine keeps its own store; the knowledge base and graph shown here are canonical
Fig. Services, and which store each one owns. Dashed arrows are writes.
ServiceRuntimeResponsibility
api-gatewayTypeScript · HonoPublic REST + TAXII 2.1. Auth, rate limiting, tenant scoping.
portalTypeScript · Next.jsTenant console and the platform-admin surfaces.
ingest-schedulerTypeScriptThe upstream connectors, on per-connector intervals.
enrichment-workerTypeScriptCanonicalise, dedupe, link, and backfill embeddings.
kb-workerTypeScriptClaims pending knowledge-base ingestion jobs.
report-runnerTypeScriptRenders scheduled and on-demand reports.
webhook-dispatcherTypeScriptDelivers subscription events, retrying with backoff.
crawler-py-scrapePythonDocument fetch and extraction for KB ingestion.
crawler-py-embedPythonEmbedding generation for the fetched corpus.

The Node / Python seam

The customer-facing application stays in TypeScript, and Python is adopted for the intel engine behind a contract rather than merging the two runtimes (ADR-0010). The intent is an independent failure domain: the engine can be down, redeployed or rebuilt without the gateway losing the ability to serve what is already in the canonical store.

The engine is not deployed yet

apps/intel-engine is in the repository and is linted and tested on every push, but it has no entry in the deployment manifest and nothing in the TypeScript services calls it. The seam described here is the design the code was written to, not a description of traffic flowing today.

Two consequences worth stating plainly. There is one Postgres, not two — the “engine owns its own store” half of ADR-0010 is not in effect. And any surface documented as reading from the engine has no source of data until it is deployed.

Which store is canonical

One Postgres with pgvector holds the canonical entities, the knowledge base and the CTI graph. There is no separate vector database, no Neo4j and no OpenSearch in the running system.

What is actually in the running stack

Postgres with pgvector carries the corpus, the embeddings, the graph and the tenant boundary. There is no separate graph database, search cluster or object store in the request path — fewer moving parts to keep consistent, and one place where row-level security applies.

Deployment

One container image serves every Node service; a SERVICE environment variable selects the entry point at start-up. That keeps a single Dockerfile and a single build for seven runtimes.

  • portal — the Next.js production server.
  • api-gateway — the public API, port from $PORT.
  • kb-worker — claims pending KB ingestion jobs on a 5s tick.
  • enrichment-worker — indicator embedding backfill.
  • report-runner — per-tenant scheduled report bundles.
  • webhook-dispatcher — outbound retry loop with backoff.
  • ingest-scheduler — the connector cadences, in-process.
Scheduling is in-process, by necessity

The host platform has no native cron, so connector cadences are setInterval inside the ingest-scheduler worker rather than external scheduled jobs. If that worker is not running, nothing ingests — and the symptom is silent staleness, not an error. The status page derives freshness from ingestion_runs precisely so this is visible.

Contracts, not prose

Two principles constrain how this is built, and both are load-bearing rather than aspirational:

  • Specs over prompts. The OpenAPI contract generates the SDK clients, so the clients cannot drift from the contract.
  • Code over prompts. Safety-critical behaviour — tenant isolation, region residency, confidence thresholds — is enforced in code and in database policy. None of it depends on a model following an instruction.
The spec currently lags the implementation

specs/openapi/v1.yaml documents 6 paths, while the gateway mounts 22 route groups. So the generated SDKs cover a subset of what the API actually serves. Treat the API overview as the current map of the surface, and the spec as the subset with a guaranteed typed client.

Architecture — OmniIntel docs · OmniIntel