# r45: colab ssh slot + http recovery

## result

implemented in `/Users/dawn/proj/albedo`; all edits are uncommitted and unstaged.
no nix rebuild, daemon restart, runtime stop/unassign, or remote file writes occurred.
**nothing here makes the change live in the running daemon.** its installed
`/nix/store/...albedo-server-1.0.0/` code is unchanged. dawn must rebuild/install
albedo and start a kernel from that updated bundle when safe. this verification
loaded the edited module explicitly into this child kernel, not into the daemon.

## what changed

- `priv/python/albedo_plugins/colab.py`: after `remote.connect()` raises
  `RemoteError`, a stdlib websocket-upgrade probe checks the ssh endpoint.
  only HTTP 429 plus `already-active-session` raises the exported
  `ColabSshSessionError` (`ColabError`, `status=429`). unrelated refusals or
  probe transport failures re-raise the original error object unchanged.
- successful connects make **no extra round trip**. the probe is on the failure
  path only, with a socket timeout capped at 10 seconds and a 4096-byte body read.
  a successful probe upgrade is immediately closed, never retained as a tunnel.
- new read-only HTTP helpers refresh proxy credentials independently of ssh:
  `runtime_status(vm)`, `contents(vm, path, content=True)`, `read(vm, path)` and
  `pull(vm, remote, local, resume=True)`. `contents(..., content=False)` returns
  metadata without loading file contents. `pull` returns the local `Path` and
  streams in 1 MiB chunks on a worker thread.
- resume validates Content-Range. an ignored Range restarts the local file
  instead of appending corrupt data. invalid ranges/HTTP errors leave the local
  file untouched; interrupted downloads retain partial data for retry.
- updated `robot-docs/colab.md`, the extension's model-facing instructions, and
  `test/harness/colab_plugin_test.py`.

## live before / after

runtime: `gpu-a100-hm-kkb-euw4a2-268zqw5nqszkj`.

ssh's symptom is **non-deterministic**, not merely unhelpful. these variants
occurred against the same occupied slot:

| observation | ssh symptom |
| --- | --- |
| dawn's earlier reproduction | `Connection closed by UNKNOWN port 65535` |
| this child's real unmodified `await colab.connect(vm, timeout=15)` | `Connection timed out during banner exchange` / `Connection to UNKNOWN port 65535 timed out` |

full error captured before editing:

```text
RemoteError: ssh target 'colab-gpu-a100-hm-kkb-euw4a2-268zqw5nqszkj' is unreachable: Connection timed out during banner exchange
Connection to UNKNOWN port 65535 timed out
```

a direct TCP/TLS stdlib websocket handshake returned:

```text
HTTP 429
{"error":"already-active-session"}
```

a real connect using the final edited source, normal installed `remote.connect`
and fresh daemon-provided credentials raised:

```text
ColabSshSessionError: Colab allows one ssh session per runtime; a previously held session was not released. This is server-side and does not clear on retry. Use the HTTP file routes to keep working: colab.contents(), colab.read(), colab.pull() (and colab.runtime_status() for liveness). stop()/unassign is the only forceful reset and destroys the VM.
```

the diagnosis comes from the endpoint, never from pattern-matching ssh stderr.
no proxy URL, token, request line, or instance dictionary is interpolated into
that message. probe transport errors are discarded in favor of the original
RemoteError; new runtime HTTP error messages omit credential-bearing URLs too.

## live HTTP verification

while ssh remained refused, the new helpers successfully read `/api/status`,
listed Jupyter's root, read file metadata, and read the text file
`NGC-DL-CONTAINER-LICENSE` (17,232 decoded characters).

real binary transfer using the final source:

```python
metadata = await candidate.colab.contents(vm, "python-apt.tar.xz", content=False)
await candidate.colab.pull(vm, "python-apt.tar.xz", local, resume=False)
```

- remote metadata size: **346,012 bytes**.
- downloaded local size: **346,012 bytes**.
- local artifact: `audits/r45_colab_ssh_slot/python-apt.tar.xz`.
- sha256: `4ab9c915d1295afbf2ca197993454d3659bb8dee4e9cd39ceba9efe5970873cc`.
- truncated only that local artifact to 12,345 bytes, then called the new
  `pull(..., resume=True)`. observed **HTTP 206**, with
  `Content-Range: bytes 12345-346011/346012`.
- resumed size and sha256 matched the original full download exactly.

all VM requests were read-only; no training files were modified.

## regression checks

- `python test/python_test_runner.py test/harness/colab_plugin_test.py`:
  **19 tests passed**. local fake HTTP endpoints exercise the actual upgrade
  request and return the specific 429, an unrelated 429, a 502, and a 101.
  only the specific 429 is relabelled. tests also cover original-error identity
  on probe network failure, no probe on successful connect, path escaping,
  credential refresh, fresh multi-chunk download, 206 resume, ignored Range,
  invalid ranges, 416/403, and truncated transfer retention.
- `uvx ruff@0.16.10 check priv/python/albedo_plugins/colab.py test/harness/colab_plugin_test.py`:
  **clean**; both Python files were formatted with the same pinned ruff.
- `uvx ty@0.0.85 check priv/python/albedo_plugins/colab.py test/harness/colab_plugin_test.py`:
  **clean**.
- `git diff --check`: **clean**.
- logs: `tests.log`, `ruff.log`, `ty.log` beside this report.
- full daemon/e2e suite and Gleam format check were not run. `gleam` is not on
  this kernel's PATH; no nix rebuild or daemon restart was attempted.

## remaining limits / activation

this names the refusal and provides a working independent path; it does **not**
release the occupied ssh slot. forceful reset via stop/unassign destroys the VM,
so it was not attempted. a later server-side reap may clear it, but retry itself
is not a reset.

HTTP paths are relative to Jupyter's root, not the ssh `/content` working
directory. resume assumes unchanged remote content; use `resume=False` for
mutable files. HTTP 416 is conservatively an error, not proof of completeness.
text reads/directory listings return their JSON/text in memory; large/binary
files belong through `pull`. HTTP recovery is file/liveness access, not a
replacement remote execution transport.

patch: `audits/r45_colab_ssh_slot/colab_ssh_slot.patch` is the complete
`git -C /Users/dawn/proj/albedo diff`. report and patch were written to `.partial`
files and renamed into place. dawn owns the commit and deployment; the running
daemon still does not have this change.


## archive note

The 346,012-byte transfer-test artifact (`python-apt.tar.xz`, an unrelated Debian package that
happened to be in the runtime's Jupyter root) was verified against the recorded
sha256 `4ab9c915d1295afbf2ca197993454d3659bb8dee4e9cd39ceba9efe5970873cc` and then deleted from this
archive rather than committed. The digest is the evidence; the bytes are noise in a git tree.