/stream is hydrant's ordered websocket event stream, and every subscriber gets a full copy of it. it's only in builds with the indexer_stream feature.

/stream isn't wire or protocol compatible with tap. tap sends events out through a sharded work queue, tracks acks on the server, and redelivers them. hydrant broadcasts every event to everyone and leaves the cursor to the client.

GET /stream #

subscribe to the ordered event stream.

query parameters #

param	type	description
cursor	integer	u64 event id to replay durable record events from, inclusive.
stream	string	the stream cursor came from (see /stream/head). if it isn't this one, hydrant sends a StreamMismatch error frame and closes the connection.

without cursor, the connection starts at the current head and only gets new events. with it, hydrant first replays the durable record events from the cursor up to the head as it was when you connected, then switches to live events.

cursor=0 replays every record event still stored. ids always go up but can skip numbers, because identity and account events use ids without being stored, and events hydrant can't render are skipped.

event envelope #

events are json text frames shaped like this:

{
  "id": 42,
  "type": "record",
  "record": {}
}

field	type	description
id	integer	u64 stream event id.
type	string	record, identity, or account.
record	object	present when type is record.
identity	object	present when type is identity.
account	object	present when type is account.

only the payload matching type is present.

record #

record events are durable, and a cursor replay can get them.

{
  "id": 42,
  "type": "record",
  "record": {
    "live": true,
    "did": "did:plc:abc123xyz",
    "rev": "3kpjxabc123",
    "collection": "app.bsky.feed.post",
    "rkey": "3kpjxabc123",
    "action": "create",
    "record": {
      "text": "hello, world!",
      "$type": "app.bsky.feed.post",
      "createdAt": "2026-05-27T12:00:00.000Z"
    },
    "cid": "bafyreihy..."
  }
}

field	type	description
live	boolean	true for live events, false for backfill.
did	string	repository did.
rev	string	repository revision, a tid.
collection	string	collection nsid.
rkey	string	record key.
action	string	create, update, or delete.
record	object	the decoded dag-cbor record, with keys in the record's cbor order (shorter first) instead of alphabetical. left out when there's no stored body.
cid	string	cid of record. left out when there's no stored body.

record and cid are left out for deletes and when hydrant doesn't store record bodies, such as with HYDRANT_ONLY_INDEX_LINKS=true.

once a redaction finishes, hydrant stops serving the body, except in events already queued for a reader.

identity #

identity events are live only, and a cursor replay doesn't get them.

{
  "id": 43,
  "type": "identity",
  "identity": {
    "did": "did:plc:abc123xyz",
    "handle": "user.bsky.social"
  }
}

field	type	description
did	string	repository did.
handle	string	current handle, left out when hydrant doesn't have one.

account #

account events are live only, and a cursor replay doesn't get them.

{
  "id": 44,
  "type": "account",
  "account": {
    "did": "did:plc:abc123xyz",
    "active": false,
    "status": "deactivated"
  }
}

field	type	description
did	string	repository did.
active	boolean	whether the repository is active.
status	string	account status, left out when hydrant doesn't have one.

websocket behavior #

• hydrant sends an empty ping every 30 seconds and answers client pings with pongs.
• clients can't send text or binary frames. hydrant closes the connection if one does.
• when /stream is in HYDRANT_THROTTLED_STREAMS (it isn't by default), a replay from a cursor runs at most HYDRANT_STREAM_REPLAY_SPEED times the bandwidth the live stream would take (10 by default), or HYDRANT_STREAM_REPLAY_FLOOR_BYTES a second (about 80 Mbit/s by default) if that's more. the live tail never waits for it.
• a subscriber that falls behind the live broadcast catches up from the database at its own pace. it never misses a record event, but it misses identity and account events from while it was behind.
• if hydrant can't read the next events from the database, it closes the connection instead of skipping them. reconnect with the last id you got as the cursor.
• when hydrant stops, it closes the connection with close code 1012 (service restart). reconnect the same way. under systemd socket activation, the reconnect waits for the next hydrant instead of being refused.
• if a send is stuck for HYDRANT_STREAM_SEND_TIMEOUT (30 seconds by default), hydrant sends this error frame and closes the connection:

{
  "type": "error",
  "error": "ConsumerTooSlow",
  "message": "stream socket send blocked for at least 30 seconds"
}

GET /stream/head #

get the id of the newest durable record event, and the stream it belongs to.

{ "id": 289455, "stream": "5b0e6c8f2a1d4e7b9c3f0a6d8e2b4c71" }

id is null while no events are stored. stream is the id of this database's stream. every database gets a fresh one, including after a wipe, and two nodes never share one. an event id only means something on the stream it came from. id is the newest event a cursor replay can deliver, not the id counter, because identity and account events use ids without being stored.

to know when a cursor=0 replay has caught up, get the head before subscribing and wait for a frame with an id at or past it. keep an idle timeout as a fallback too, because if the head event itself can't be rendered, it's skipped and never arrives.

chunks #

with HYDRANT_STREAM_CHUNK_SPAN set, hydrant also seals the stream into immutable files that a subscriber starting from scratch can read instead of replaying /stream from 0. chunks are compressed and cacheable, and serving one is just a file read instead of rendering every event again for each subscriber.

each chunk covers the ids first through last and holds every record event in that range. like a replay, it has no identity or account events. reading the chunks in order and then /stream from the id after the last one gets you exactly what /stream?cursor=0 would. the file layout is under chunk format.

GET /stream/manifest #

list the chunks to read, in order.

{
  "stream": "5b0e6c8f2a1d4e7b9c3f0a6d8e2b4c71",
  "generation": 41,
  "through": 917503,
  "chunks": [
    { "first": 786432, "last": 851967, "events": 47512, "bytes": 5104330, "sha256": "9f2c..." },
    { "first": 851968, "last": 917503, "events": 61664, "bytes": 11420553, "sha256": "41ab..." }
  ]
}

(the list is shortened here.) the chunks run from id 0 through through with no holes. while hydrant is rebuilding a chunk, the list and through stop just before it. read all the chunks, then connect to /stream?cursor={through + 1}&stream={stream}. if a chunk fails partway through, you don't have to start over. connect to /stream from the id after the last event you got. generation changes whenever the list does. returns 404 when chunks are off.

GET /stream/chunk/{sha256} #

get a chunk's bytes as stored, with content type application/zstd. its sha256 is both the name and the ETag. chunks never change, can be cached forever, and support Range requests. after a merge, the old chunks stay readable for 15 minutes for subscribers still working through an older list. a chunk hit by a redaction or erase returns 404 right away.

how the list changes #

a new chunk is one span long. whenever the newest 4 chunks are the same size, hydrant merges them into one, up to 16 spans. past that, the list grows by one chunk every 16 spans, which gives a subscriber several chunks to fetch in parallel. hydrant builds chunks on a low priority thread and seals only one span every 30 seconds while ingest is behind. building chunks for a whole stream (after turning chunks on, or after a wipe) mostly waits for ingest to catch up.

a redaction or erase hits every chunk that has events for that repository. those chunks drop out of the list until hydrant rebuilds them from the event rows without the removed bodies, matching what a replay sends. the stream has no redaction event, and a subscriber that already read an affected chunk keeps what it got, just like one connected to /stream at the time.

chunk format #

a chunk is a STAR archive compressed with zstd. the file has several zstd frames, and some decoders stop after the first one unless you tell them to read across frames. the archive's header meta is a DRISL map with these keys:

key	value
$type	"systems.gaze.hydrant.streamChunk"
stream	the stream id, as in /stream/head
first	the first id the chunk covers
last	the last id the chunk covers

events #

each pair is one event, in id order between first and last. the key is the event id as 8 bytes big-endian, and the value is a DRISL map of the event's fields, with the record as hydrant stores it:

key	DRISL	/stream field
live	bool	record.live
did	text	record.did
rev	text	record.rev
collection	text	record.collection
rkey	text	record.rkey
action	text	record.action
record	byte string, left out when there's no body	record.record
cid	tag 42 cid link, left out for deletes and when it matches the cidv1 (dag-cbor, sha2-256) of record's raw bytes	record.cid, as a base32 string

as a /stream frame #

a chunk event has the same fields as a /stream record frame. you only need this part to rebuild the exact json /stream sends.

the json frame decodes record's dag-cbor bytes into json. maps keep their cbor key order, a tag 42 link becomes {"$link": "<base32 cid>"}, bytes become {"$bytes": "<padded base64>"}, and ints, strings, bools, nulls and arrays stay as they are. if a body can't be represented that way without losing information, hydrant serializes it the way jacquard's RawData does instead. that happens when a body has a float, an int past 64 bits, a $link or $bytes map key, keys out of dag-cbor order or repeated, or a blob map with any field besides $type, ref, mimeType and size. if a body isn't valid dag-cbor at all, record is left out of the json frame. the whole frame is

{"id":<id>,"type":"record","record":{"live":..,"did":..,"rev":..,"collection":..,"rkey":..,"action":..[,"record":<body>][,"cid":"<cid>"]}}

with no whitespace, byte for byte what /stream sends.