# svfe-pages
`svfe-pages` prerenders [`svfe`](https://git.klbr.net/ptr.pet/svfe)'s repo and user pages into static html for a [yumak](https://git.klbr.net/ptr.pet/yumak) (`knot-static`) generation, so pages work without js and hydrate into the `svfe` spa.
renders svfe's repo pages for a knot-static generation, so they work without js and hydrate
into the svfe SPA. it runs as knot-static's `[site_hook]`, reads only the knot's public xrpc
surface, and re-renders just the pages the generation's xrpc diff touches.
- **incremental**: runs as a `knot-static` `[[hooks]]` entry after each activation, reads `<generation>/hook.json` and `xrpc-diff.jsonl`, and re-renders only the pages whose `/xrpc/...` reads changed.
- **deduplicated**: stores a repo's sidebar once per content at `<repo>/_shared/<hash>.json` so sidebar edits don't re-render every page, and writes open issue/pull counts to `<repo>/_latest/counts.json` so new tickets rewrite one file instead of the repo.
- **bounded**: caps each repo by page count, commit count, and disk bytes (`budget.mjs`). routes past the budget fall back to the `svfe` spa (`<repo>/_latest/routes.json` tells the client which routes have prerendered documents).
- **zero-downtime across deploys**: copies each rendering `svfe` build's `_app/immutable/` chunks into `<out>/www` and keeps older builds' chunks until no served page references them.
svfe-pages <hook.json> --server <svfe build-server> [--client <svfe build>] --out <dir>
[--read <xrpc base>]
[`git.klbr.net`](https://git.klbr.net) runs `svfe-pages` and hosts this repo (`did:plc:qaxvrxxd6ubibxniuzaktvon`).
see `nginx-site.conf` for serving `<out>` in front of the SPA, and `budget.mjs` for what a
repo may cost on disk.
## usage
a page loads the client chunks of the svfe build that rendered it, so `--client` (by default
the `build` next to `--server`) has its `_app/immutable` copied into `<out>/www` before any
page renders, and a build's files are removed once no served page was rendered by it. a
deploy then never leaves a served page without its chunks, while the re-render runs.
```
svfe-pages <hook.json> --server <svfe>/build-server [--client <svfe>/build] --out <dir> [--read <xrpc url>]
```
what every page of a repo shows the same way, the sidebar, is stored once per content in
`<repo>/_shared/<hash>.json` and isn't a page's own read, so a change to it re-renders
nothing. a page kept from an older render names an older copy, so check an incremental run
against a fresh one of the same generation with
```
--server <dir> svfe's build-server/ directory
--client <dir> svfe's build/ directory (defaults to ../build next to --server)
--out <dir> output directory (serves from <out>/www)
--read <url> local knot url to fetch /xrpc/... from (e.g. http://127.0.0.1:8080)
--workers <n> parallel repo worker processes (default: half of cpu cores)
--jobs <n> pages in flight per worker (default: 6)
--max-pages <n> max prerendered pages per repo (default: 1000)
--max-commits <n> max prerendered commit pages per repo (default: 60)
--max-repo-bytes <bytes> max gzipped bytes on disk per repo (default: 20 MiB)
--max-total-bytes <bytes> max gzipped bytes across all repos (default: 2 GiB)
--max-blob-bytes <bytes> max blob size to prerender (default: 128 KiB)
--full ignore cached state and re-render every page
```
node verify.mjs <incremental out> <fresh out>
## nixos example
the newest sidebar of each ref is also at `<repo>/_latest/sidebar/<ref>.json`, for a page
the SPA renders without a prerendered copy to name, and the routes the repo has pages for at
`<repo>/_latest/routes.json`, so the SPA loads only those as documents. the repo's open ticket
counts, which every page's tabs show, are read apart from the pages into
`<repo>/_latest/counts.json`, so a new ticket rewrites that file and no page.
see [`nginx-site.conf`](nginx-site.conf) for the standalone nginx locations, or wire `svfe-pages` and `svfe` into `services.knot-static` on nixos:
each owner of a repo also gets a user page, `<owner>.html` beside the owner's repos, with
`<owner>/_latest/routes.json` naming it, rendered again when a read it made changes (the
profile, the handle, the repo list) and moved when the handle changes.
```nix
let
system = pkgs.stdenv.hostPlatform.system;
svfe = inputs.svfe.packages.${system}.svfe-web;
svfe-pages = inputs.svfe-pages.packages.${system}.svfe-pages;
knot = config.services.knot-static;
inner = "http://${knot.listen}";
pages = "/var/lib/svfe-pages";
in
{
services.knot-static = {
enable = true;
hostname = "git.klbr.net";
settings.index.links = "https://git.klbr.net";
settings.hooks = [
{
name = "svfe-pages";
command = "${pkgs.coreutils}/bin/nice";
args = [
"-n" "19"
(lib.getExe svfe-pages)
"--server" "${svfe}/build-server"
"--out" pages
"--read" inner
"--workers" "4"
"--max-repo-bytes" (toString (64 * 1024 * 1024))
"--max-total-bytes" (toString (8 * 1024 * 1024 * 1024))
];
}
];
nginx.virtualHost.locations = {
"/" = {
proxyPass = null;
root = "${pages}/www";
extraConfig = ''
gzip_static always;
gunzip on;
if ($arg_page ~ "^0*([2-9]|[1-9][0-9]+)$") {
set $page $1;
rewrite ^(.*)$ "$1~page-$page?" last;
}
add_header Cache-Control "no-cache";
try_files $uri.html $uri/index.html @knot;
'';
};
"@knot" = {
proxyPass = inner;
extraConfig = ''
proxy_intercept_errors on;
error_page 404 = @spa;
'';
};
"@spa" = {
root = "${svfe}/build";
extraConfig = ''
gzip_static on;
try_files $uri /index.html =404;
'';
};
"^~ /_app/immutable/" = {
root = "${pages}/www";
extraConfig = ''
gzip_static on;
expires max;
try_files $uri @immutable;
'';
};
"@immutable" = {
root = "${svfe}/build";
extraConfig = ''
gzip_static on;
expires max;
'';
};
"~ ^/[^/]+(/[^/]+)?/_latest/" = {
root = "${pages}/www";
extraConfig = ''
gzip_static always;
gunzip on;
default_type application/json;
add_header Cache-Control "no-cache";
try_files $uri =404;
'';
};
"~ ^/[^/]+/[^/]+/_shared/" = {
root = "${pages}/www";
extraConfig = ''
gzip_static always;
gunzip on;
default_type application/json;
expires max;
try_files $uri =404;
'';
};
};
};
systemd.tmpfiles.rules = [ "d ${pages} 0750 ${knot.user} ${config.services.nginx.group} -" ];
}
```
## tests
```sh
node --test
node --test
node verify.mjs <incremental out> <fresh out> # check an incremental render against a clean one
```