nemotron/ultra/test: stop aiperf runs from overwriting each other's artifacts

[dmvevents]

What

test/test-aiperf.sh wrote every run to ${MODEL_PATH}/aiperf/${DEPLOYMENT_TYPE}. That path is not unique per run:

• all topologies in a folder share one DEPLOYMENT_TYPE, so a disagg/lws-2pp run and the disagg/lws-pp2 run after it land in the same directory
• a repeat of the same topology (a cold run, then a warm rerun) lands there too

aiperf truncates profile_export.jsonl when it starts, so the earlier run's per-request records are gone before anyone reads its summary, and the summary itself is overwritten when the later run finishes.

Observed

p6-b200, 2026-08-16. A completed disagg/lws-2pp NVFP4 report — profile_export_aiperf.{csv,json} and server_metrics_export.{csv,json}, benchmark_id 0901ab86a8a7, ended 02:01:37Z — sat in .../NVFP4/aiperf/disagg/ next to a zero-length profile_export.jsonl dated 03:38, written by the disagg/lws-pp2 run started 97 minutes later. The per-request records needed to order requests by start time (the check that separates a first-wave JIT tail from a steady-state one) were already unrecoverable, and the summary would have gone the same way when the second run finished.

Comparing topologies, which is the point of having five of them, destroys the evidence it is comparing.

Fix

Artifacts go to ${MODEL_PATH}/aiperf/${DEPLOYMENT_TYPE}/${MANIFEST_TYPE}/${AIPERF_RUN_ID}, printed before the run starts:

Artifacts: /shared/models/hf/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4/aiperf/disagg/lws-2pp/20260816T040204Z

AIPERF_RUN_ID defaults to a UTC timestamp and takes any label (AIPERF_RUN_ID=cold) — same idiom as AIPERF_WARMUP_REQUESTS and WARMUP_ENABLED.

test/.env gains MANIFEST_TYPE, mirroring agg/.env and disagg/.env. Nothing in test/ renders a manifest, so it only labels the results folder. DEPLOYMENT_TYPE becomes ${VAR:-default} for the same reason it did in agg/.env in #74: selecting disagg for a benchmark should not require editing a tracked file.

3 files, README documents the path and both knobs.

Gates (all cluster-free)

• bash -n on test/.env and test/test-aiperf.sh
• artifact path rendered for all four disagg topologies and for two run ids of one topology — confirmed distinct
• defaults with no exports still resolve to agg / deployment / BF16
• test-aiperf.sh stub-executed against a PATH-shimmed kubectl in three configurations: NVFP4 disagg lws-2pp emits --warmup-request-count 30, BF16 agg default emits none, AIPERF_RUN_ID=cold with AIPERF_WARMUP_REQUESTS=0 honours both

Not in this PR

The lws-2pp NVFP4 tail itself. That run used both warmup layers — the manifest warmer and --warmup-request-count 30 (3 waves at --concurrency 10, excluded from the report, confirmed in the export's run_info) — and still reported TTFT p90 100,539 ms against a p50 of 742 ms. Its server_metrics_export.csv shows the two decode replicas measurably asymmetric: dynamo_frontend_worker_last_inter_token_latency_seconds max 4.44 s on one worker, 62.99 s (p99 62.63 s) on the other. That points at warmup coverage on replicas: 2 behind a round-robin frontend, which the template's own comment already flags — but those pods are gone, so it is a hypothesis, not a verified finding, and it does not belong in a fix PR yet.