AICamera

openspec/changes/archive/2026-09-10-add-parakeet-api/reviews/2026-09-10-advise.md

Advise: add-parakeet-api

ADVISE: send-back READER: agt-fable-5-1-arch-review SPAWN: /Users/dukejones/work/ClientProjects/AllSystemsGo/AICamera/.spawns/aicam-parakeet-shape-1789079544-12885-077901a4

Rigor: architecture. Density: standard. Reader family: Claude (Fable 5.1). Author family: grok. Read order: packet Why → change spec specs/parakeet-asr/spec.md → package.json (adapter-static) → sim/3090/serve.py (bind_ip, port 8745) → daBOM src/lib/server/api/root.ts (well-known trio) → daBOM src/lib/server/api/app.ts (app.doc31) → Papyrus morphist-asr/src/runtime/backend.rs (recommend_for_asr). Blind take written before proposal.md, steer.md, design.md, tasks.md were opened. No living parakeet-asr spec exists; capability is ADDED at fold.

Blind take (written before design.md)

  1. Pin. The OpenAPI document SHALL carry servers with the /api/v1 base. daBOM’s well-known route is api.request('/openapi.json'), so the bytes are identical, but an agent that reads /.well-known/openapi.json and sees paths./transcribe with no servers will POST to /transcribe at root and get 404. Same bytes is necessary, not sufficient; the document must be self-locating.
  2. Pin. Sidecar boundary. Bun does not run Parakeet TDT decode natively; the warm engine is a second process (Python + onnxruntime CPU or sherpa-onnx). The spec must say the sidecar listens on loopback or a unix socket only, and that the Bun process is the sole tailnet listener. Otherwise “Tailscale bind” is true of one process and a second unguarded port appears on the same box.
  3. Pin. CPU-only must be mechanical, not aspirational: CUDA_VISIBLE_DEVICES= (empty) in the unit env, the CPU onnxruntime wheel not onnxruntime-gpu, and provider list hard-set to CPUExecutionProvider. Papyrus recommend_for_asr classes the 14600KF (Raptor Lake, AVX2+FMA, no AVX-512) as Avx2Fma → CPU; the decision here is already made, so cite the rule and pin the env, do not port the detector.
  4. Pin. CPU contention is the real cost, not VRAM. The 14600KF is 6P+8E and the same box runs the 30 fps compositor’s host-side work. A Parakeet session with default intra-op threads takes every core. The spec needs a thread ceiling (intra-op threads ≤ N, e.g. 4) and a scenario: transcribe while the 30 fps plugins run and frame time does not regress. “3090 stays free” without a CPU budget just moves the stall.
  5. Pin. Weights are a unit artifact, not a first-request download. Pin the INT8 ONNX bundle path (sherpa-onnx parakeet-tdt-0.6b-v3-int8 or NeMo export) and a checksum in the unit; the “warm sidecar serves the second call” scenario is meaningless if the first call is a multi-hundred-MB fetch.
  6. Pin. Stream operation shape. @hono/zod-openapi createRoute can declare application/octet-stream request + application/x-ndjson or text/event-stream response, but not both response types on one operation. Pick one (NDJSON: symmetric with the PCM request, no browser SSE constraints). State that v1 stream clients are non-browser (curl, Bun, Node): streaming request bodies over fetch need HTTP/2 + duplex: 'half' which browsers do not do. End-of-stream is connection close; the last event carries final: true.
  7. Pin. Probe. 503 on sidecar-unavailable is stated but unobservable before a clip is posted. Add GET /api/v1/health in the OpenAPI document reporting sidecar warm | cold | down, provider (CPUExecutionProvider), and model id, so the 503 scenario and the CPU-only scenario both have a cheap check.
  8. Refuse. Any Hono/zod dependency in the root package.json. The Pages build is vite build with adapter-static; the API must live in its own directory with its own package.json and lockfile so the static site’s dependency graph does not grow a server. Root already shows bun.lock untracked and package-lock.json deleted; do not let this change be the reason hono lands at root.
  9. Refuse. Resampling or channel-mixing in v1. Wrong shape is 422, full stop. The moment the API resamples 44.1k stereo “to be helpful”, the 422 scenario is dead and the CPU budget in (4) is unknowable. WAV with a non-audio content-type is 415, not 422.
  10. Tradeoff. Python sidecar vs Bun-native ONNX (onnxruntime-node). Bun-native is one process and no protocol, but TDT greedy decode + SentencePiece tokenizer must be re-implemented and maintained. Python (sherpa-onnx) is the supported Parakeet path today, costs a second process and a loopback protocol, and is what Papyrus deliberately did not do because it is Rust-native. Take the sidecar: the Bun process is the contract, the engine is swappable, and the “do not vendor” constraint stays honest.

Compare against proposal.md, steer.md, design.md, tasks.md

Also read after the take: sibling changes add-parakeet-engine, add-parakeet-hono, add-parakeet-stream (all ADD to parakeet-asr), daBOM app.ts:657 (servers: [{ url: '/api/v1' }]).

Shape matches the take. Standalone Bun + Hono + @hono/zod-openapi on fractal1, :8750 beside bay-feed :8745, Tailscale IPv4 only, daBOM well-known trio, warm Python onnx-asr INT8 sidecar, CPU EP with CUDA_VISIBLE_DEVICES= and four intra-op threads, windowed TDT for stream, no vendoring of morphist-asr. The steer decided the same five things the take would have decided. This change is the shape node; engine, hono, and stream do the writes, so a three-line tasks.md is correct, not thin.

Answered by the author (take → where):

  • (2) sidecar boundary → design: --stdin/IPC-->; no second port. Correct choice.
  • (3) mechanical CPU-only → design: CUDA_VISIBLE_DEVICES= + providers=["CPUExecutionProvider"].
  • (4) thread ceiling → design: “Four intra-op threads.” Engine proposal repeats it.
  • (5) weights as artifact → engine tasks: ~/aicam/parakeet-cpu INT8 weights on fractal1.
  • (8) not in root package.json → engine tasks: asr/ or sim/parakeet/. Directory undecided but off-root.
  • (10) sidecar over Bun-native → steer: Python onnx-asr sidecar, decided.

Not answered, and each is a line in the spec that gets folded living. design.md and tasks.md do not fold; specs/parakeet-asr/spec.md does. Four of the author’s good answers live only in design, and one design line contradicts the spec.

  1. Design contradicts spec on resampling. Spec, Clip transcribe: “Wrong sample shape SHALL be 422.” Design, Audio: “Other rates resample in the sidecar or 422.” The engine actor reads design. If the sidecar resamples, the 422 scenario is dead and the four-thread budget is unknowable. Strike “resample in the sidecar or” from design. Spec wins; say so.
  2. Health is promised in the journey and absent from the spec. Proposal, Empty: “model not loaded yet. Health says so; transcribe is 503, not a hang.” No requirement produces a health operation. Without it the 503 scenario is only observable by posting audio. Add GET /api/v1/health to the Well-known or Clip requirement: reports sidecar warm|cold|down, provider, model id, and is listed in the document.
  3. Sidecar listener is undecided across the DAG. Design says stdin/IPC. Engine tasks say “systemd unit Tailscale-adjacent to bay-feed, or stdin protocol the Bun process owns.” Tailscale-adjacent means a second tailnet port that the Tailscale-bind requirement does not cover (it names “the ASR process”, singular). One sentence under Tailscale bind: the sidecar SHALL NOT open a network listener; it is a child of the Bun process on stdin/IPC. That closes the engine “or” from the spec that owns the boundary.
  4. Thread ceiling and no-CUDA env are design-only. CPU-only Parakeet requirement says CPUExecutionProvider and “SHALL NOT create a CUDA context” but not how. Add: the sidecar SHALL run with CUDA_VISIBLE_DEVICES empty and intra-op threads ≤ 4. The GPU scenario is already good; add one CPU scenario: transcribe while bay-feed’s 30 fps stack runs, frame time does not regress. That is the actual cost on this box, and the spec is silent on it.
  5. servers is the difference between “same bytes” and “self-locating”. daBOM sets servers: [{ url: '/api/v1' }]; “copy daBOM” implies it, but the hono sibling’s scenario accepts ”/transcribe or /api/v1/transcribe as declared”, which is exactly the ambiguity an agent trips on. Well-known requirement: the document SHALL carry servers with the /api/v1 base so paths keys are relative to it. One line.

Notes, not blocking.

  • NDJSON or SSE: spec, design, steer, and stream sibling all say “NDJSON/SSE”. Recommend NDJSON: symmetric with the PCM request, no SSE framing, and browser SSE ergonomics do not matter because streaming request bodies over fetch need HTTP/2 + duplex: 'half' that browsers do not do. State v1 stream clients are non-browser. If the author keeps “as declared” that is acceptable; the document is the contract.
  • Four changes ADD to one new capability. This change carries the full spec; engine, hono, stream carry narrower requirements that overlap it (engine “Warm CPU sidecar” ⊂ api “CPU-only” + “Clip transcribe”). Fold this change first and fold the siblings as scenarios under its requirements, not as parallel requirements, or the living spec says the same thing three ways.
  • Hardcoded default 100.103.147.70 in the spec is fine as a default; the unit should resolve like serve.py bind_ip() (env → tailscale ip -4 → default) and never fall through to 0.0.0.0. Design already says never 0.0.0.0.
  • WAV with a non-audio content-type is 415, not 422. Reserve 422 for declared-but-wrong shape.

Verdict. Send-back. The architecture is right and every decision in the steer holds. The send-back is five spec lines and one struck design clause, so the living spec carries what the design already knows. Re-advise should be short.