Ingestion.

Ten upstream connectors on independent cadences, each run recorded whether it succeeds or not.

A single run

Fig. One connector run: cadence, fetch, normalise, upsert — plus the run row written on every attempt.

The connectors

Connector	Gives us	Entity
`abusech-urlhaus`	Malware distribution URLs	indicators
`abusech-threatfox`	IOCs with malware attribution	indicators
`abusech-malwarebazaar`	Malware samples and hashes	indicators · malware
`alienvault-otx`	Community pulses	indicators · actors
`cisa-kev`	Known exploited vulnerabilities	vulnerabilities
`epss`	Exploit prediction scores	vulnerabilities
`github-ghsa`	Advisories for packages	vulnerabilities
`nvd-cve`	The CVE corpus with CVSS	vulnerabilities
`mitre-attack`	Techniques, tactics, groups	attack_patterns · actors
`ransomware-live`	Ransomware victim postings	actors · indicators

Cadence

Each connector has its own interval, because the upstreams move at different speeds — KEV changes a few times a week, URLhaus continuously. Intervals live in the ingest-scheduler worker as in-process timers.

No external scheduler

The host platform has no native cron, so this is setInterval inside a long-running worker rather than a scheduled job. If that worker is not running, nothing ingests — and the failure is silent staleness rather than an error anyone gets paged for.

That is exactly why freshness is derived from ingestion_runs and surfaced on the public status page: it makes a stopped scheduler visible instead of invisible.

What a run records

The run row is opened before the fetch, so a process that dies mid-request still leaves evidence that the attempt happened.

started_at — when the attempt began.
A status: succeeded, partial, or failed.
Counts of records read and written.
The error, when there is one, truncated to a sane length.

A failed run is non-destructive: previous records stay in place. A source that fails for a week looks stale, not empty — which is the honest representation of what happened.

Your own intel

Beyond the upstream connectors, a tenant can push its own indicators over POST /v1/private/submit. Those are written tenant-scoped and never merged into the global corpus.

Up to 500 indicators per call.
Per-indicator verdict, confidence, note, tlp and tags.
A default_tlp for the batch, so markings do not have to be repeated.