7.4 kB 69 lines
393a2a86
round 39: port serving fixes and PLE reuse; verify parity, record perf uncertainty
dawn
⋯
audits/r39_serve.md
sourcerendered
download raw history
at main Markdown

round39 serve + tokenizer port #

scope and source #

  • upstream range: 296b59d2f73c8f04a805df29c2fa97655adace78..477ac13cbe8e582c20484871ce459d20d82df09d.
  • read vendor/strata/AGENTS.md, docs/DETAILS.md thinking/API sections, docs/LOGIT_BIAS.md, design-preflight and orchestrating-agents.
  • owner boundaries: tokenizer owns exact incremental encoding; service owns request preparation/cache admission, thinking budgets and API defaults; telemetry owns fail-open platform readers. no engine/model/GPU execution or running service changes.
  • all 21 upstream serve/tokenizer changed files ported. 16 match upstream exactly; exceptions: server.py retains the pre-existing local image omission hunk unchanged and caps prompt retention; logit_bias.py removes unsafe identity caching; test_logit_bias.py adds regression coverage and removes obsolete _last resets; telemetry.py caches/releases the working-set device; test_server.py covers those hardening changes. one local test file added.
  • complete owned patch, including untracked upstream tests: results/round39/serve-owned-final.patch.
  • sha256s of every owned final file and upstream counterpart: results/round39/serve-source-hashes.json.

behavior and deviations #

ports prompt encoder resume points, plain-span-aware prefix reuse and rejected-prompt forgetting; parser wrapped-call scan cursor and bounded suffix checks; bounded content-keyed logit-bias LRU; apple IOKit telemetry; anthropic hard thinking budgets and automatic answer reservation. also carries upstream admission ordering (opt-in), turn-aware slot choice, sampling/model-card/web defaults, scrape-window throughput gauges, config diagnostics, and prompt telemetry fields.

local image omission still mutates content lists before image processing and encoding, exactly as before. new tests prove retained text-part identity, omitted images cached as text, full/incremental token parity, repeated fresh/mutated-object requests, disabled-switch error, and no mutation when vision exists.

parent review identified upstream's _last identity+length fast path returning stale values after same-length mutation. removed it rather than asserting request immutability. every large input now receives a JSON content key; four-entry bounded LRU and cached engine-key strings remain. new tests cover nested list/dict edits to valid/invalid values and JSON-equivalent inputs/duplicate IDs. red-on-upstream evidence: serve-logit-bias-upstream-repro.log (one test, two failing subtests); fixed bias suite: 14 passed.

hardening requested after opus review: service retention is capped at the explicit PROMPT_ENCODER_KEEP_MAX = 16, including huge engine INFO and lazy-start argument values. apple working-set budget is read once per _Metal instance and reused by read and static telemetry. the local SDK's /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Metal.framework/Headers/MTLDevice.h:130 declares MTLCreateSystemDefaultDevice NS_RETURNS_RETAINED; the helper now releases it in finally, including unsupported-selector returns. mock tests count one helper call through three reads and Telemetry static initialization, and verify device release on success/unsupported selectors.

validation #

python: /tmp/r39-serve-venv/bin/python (3.14.7); dependencies: results/round39/serve-python-packages.txt. initial system python lacked jinja2/regex; initial import-failure logs retained separately. baseline tree copied to /tmp/r39-serve-baseline, restoring only owned paths to source baseline plus original local image hunk; shared checkout never reverted.

  • tokenizer baseline: 6 passed.
  • tokenizer default candidate: 11 run, 3 skips (real-vocabulary classes/oracle unavailable by default).
  • tokenizer with STRATA_TOKENIZER=$PWD/models/K256_reap_l2_sum/tokenizer: 18 run, 1 skip (external llama.cpp oracle vectors absent), no failures; reads vocabulary only, no model execution.
  • image compatibility: 3 passed.
  • final logit bias: 14 passed.
  • hardening affected suites: 39 passed in 3.786s (serve-hardening-affected.log), with real vocabulary enabled; rerun after final readability adjustment also passes 39 (serve-hardening-final.log).
  • first full serve baseline/candidate attempts hit 300s deadlines; partial logs retained as serve-baseline.log / serve-post.log. no completed suite count claimed for these.
  • completed full baseline: 785 tests in 494.236s, 1 failure, 11 skips (serve-baseline-full.log).
  • completed full candidate (pre-hardening imports, before one extra bias-key test): 852 tests in 516.658s, 1 failure, 6 skips (serve-final-full.log).
  • the sole failure is identical on baseline/candidate: test_keepawake.KeepAwakeTests.test_service_holds_for_the_whole_request_including_early_close, line 87. this is an existing asynchronous-observation race, not a port regression: the mock request can acquire and release its hold before the keep-awake daemon acquires _cond; _loop then sees _count == 0 with held == False and waits without emitting either transition. the test assumes the daemon observed the transient busy state. deterministic baseline/candidate reproductions block that condition lock through the request and both record count=0, calls=[], then only idle reset on close (keepawake-mechanism-{baseline,candidate}.log). no timeout increase, expectedFailure, or unrelated product/test edit applied.
  • final hardening is checked by the separate 39-test affected-suite run, not falsely folded into the older full-run count.

changed files #

  • vendor/strata/serve/frontend.py
  • vendor/strata/serve/logit_bias.py
  • vendor/strata/serve/runconfig.py
  • vendor/strata/serve/server.py
  • vendor/strata/serve/telemetry.py
  • vendor/strata/serve/test_admit_order.py
  • vendor/strata/serve/test_effort_paths.py
  • vendor/strata/serve/test_logit_bias.py
  • vendor/strata/serve/test_model_card_sampling.py
  • vendor/strata/serve/test_parallel.py
  • vendor/strata/serve/test_parser_call_scan.py
  • vendor/strata/serve/test_parser_hold.py
  • vendor/strata/serve/test_prompt_encoder.py
  • vendor/strata/serve/test_q36_config.py
  • vendor/strata/serve/test_reasoning_loop_recovery.py
  • vendor/strata/serve/test_server.py
  • vendor/strata/serve/test_slot_alloc.py
  • vendor/strata/serve/test_web_sampling.py
  • vendor/strata/serve/web/app.js
  • vendor/strata/tools/strata_tokenizer.py
  • vendor/strata/tools/test_strata_tokenizer.py
  • vendor/strata/serve/test_image_omission.py

final review #

view_diff reviewed every tracked owned diff; view_code reviewed every new upstream/local test file and final local mutation/key-equivalence tests. tokenizer reuse helpers are shared with service rather than duplicated. upstream comments/style retained for port fidelity. no assertion or test removed except obsolete _last reset setup lines; upstream loop-recovery tests explicitly disable automatic answer_room to continue testing the original loop-guard path.

parent independent verification #

parent reports an independent full-suite run passed: 852 tests in 517.636s, 11 skips, exit 0; its independent hardening selection also passed 39/39. these are parent-reported results, not this agent's own runs. the baseline-shared keepawake race above did not reproduce in the parent full run. no additional suites launched after parent began quiet paired CLI benchmarking.