[plotarmordev]

Summary

grill-perf media can now benchmark a running OpenAI-style speech server (POST /v1/audio/speech), so a text-to-speech recipe can require contributors to attach a Grill A/B/A2 result for any speed claim.

• Runner media-speech-server-v1: declares the endpoint (plain HTTP only to a literal loopback address, otherwise HTTPS), model, optional voice, response_format (wav or pcm) and stream. The workload is audio, with input texts of at most 4096 characters.

• Capture: serial requests against an already-running server; the Grill never starts or stops it. Each sample records:

  • end_to_end_us and, for streamed replies, first_audio_us;
  • audio duration from the bytes, and realtime_factor;
  • status and headers;
  • the body, with its sha256.

  Non-200 replies, finish reasons other than stop, partial frames, timeouts and transport failures are kept and marked ineligible with specific reasons, never retried. A missing X-Finish-Reason is recorded as unknown and counted, since plain OpenAI servers don't send it.

• Compare:

  • first_audio_us is gateable only when every arm streams; end_to_end_us stays gateable;
  • the new duration_ratio compares audio length against the reference arm, with both directions adverse, so truncated or stretched speech can't pass as faster;
  • byte equality across arms is reported as a fact.

• Inspect: rechecks bodies and headers offline.

• Docs: a MEDIA.md section with a ready-to-copy example; RELEASE-NOTES Unreleased.

Testing

• cargo fmt --check, cargo clippy -p grill-perf --all-targets -- -D warnings, unit tests (149), media CLI tests (109), and the full grill-perf suite in round 1.
• Loopback fixture server:
  • whole WAV, streamed PCM, a 422, finish reason length or missing;
  • a partial frame, a timeout, and offline tamper rejection;
  • A/B/A2 PASS on first audio, plus duration_ratio regressions for halved and doubled audio.
• Compatibility: v0.10.0 CLI audio captures still inspect, with the same evidence digest.
• Review: built by WRITER and reviewed blind by SIX, which approved round 2. No live speech server was run.

Performance claim

None.