[RFC] GDN hybrid decider kernels: close HF→vLLM/SGLang gap (causal_conv1d + FlashInfer/Triton GDN) #141

Context (from system1-omni)

Owner asked to turn a live perf gap into an InferRSI problem and push it here.

On L20X, system1-omni draft integrations for PPLX-class 27B and Clef-class GDN hybrid deciders show C1 latency dominated by HF backbone forward—not frontend/HTTP/weight IO.

Measured baselines (Public231 C1, L20X, concurrency 1, 231/231; no-thinking where stock thinking 422s)

Profile sinks (PPLX @ 1ba07d15, in-process ~136 tok)

• Total ~89 ms; forward CUDA ~87.7 ms (~98%); prepare ~1.4 ms; parse/post/answer <0.3 ms
• Short-seq floor ~88–91 ms (nearly flat 123–275 tok); p95 driven by long prompts (fwd ~327 ms @ ~2676 tok)
• Missing causal_conv1d → transformers fallback warning → generic depthwise F.conv1d in trace
• Path today: HF-SDPA + GDN, no CUDA graphs
• Param ratio vs StartLux (~6.75×) under-explains the ~12–14× latency gap

Tips pending remeasure (do not treat as gains yet): PPLX 7233620 (gate on causal-conv1d 1.7.0 + sm_89 cubin); Clef 033685b7 (FLA + causal_conv1d).

Research question

Can we close the kernel gap between our HF-serve path and the vLLM/SGLang stack for the same GDN hybrid family, with only verified Public231 C1 gains on L20X (sm_89)?

Reference stack (owner direction):

• Runtime stays ours (not “replace with vLLM/SGLang wholesale”)
• Full-attention layers: FlashInfer (or Triton/FA as fallback)
• GDN linear-attention: Triton default / FlashInfer optional (see SGLang --linear-attn-* / gdn-backend; vLLM FlashInfer GDN prefill/decode work)
• Causal convolution: causal_conv1d as a required separate op (not FlashInfer); fail ready if fallback to F.conv1d

Hypotheses (falsifiable)

1. Installing a correct sm_89 causal_conv1d and binding it (no fallback warning) yields a measurable C1 p50 drop on PPLX/Clef vs the baselines above.
2. Further GDN path work (FlashInfer or FLA/Triton GDN kernels, short-seq graphs) only pays if profile shows GDN/conv still in the top sinks after (1).
3. Classification C1 (no decode) has a different optimal kernel mix than decode-heavy vLLM/SGLang benches—document what does not transfer.

Evidence loop (keep only verified gains)

1. Profile → tip → L20X remeasure (warmup 5, c=1, Public231) → keep tip iff p50/p95 improve without quality collapse.
2. AB/BA or remeasure-gate; no “samples=0” commits counted as wins.
3. Record which kernels actually bind on sm_89 and the install/gencode recipe.

Success criteria

• Documented mapping: vLLM/SGLang GDN+conv1d ops ↔ our HF worker path
• Reproducible L20X install for causal_conv1d (sm_89) with zero fallback warning on ready
• At least one tip with verified C1 p50/p95 delta vs table baselines
• Knowledge note: what transferred from vLLM/SGLang vs what did not for classification C1

Non-goals

• Merging system1-omni PRs without separate owner yes
• Replacing system1-omni serving with vLLM/SGLang
• Publishing local infra details in group chats

Related

• system1-omni drafts: #164 StartLux, #165 Clef, #166 PPLX
• Skills draft (optimize-remeasure-gate): #172
• Upstream refs: vLLM FlashInfer GDN prefill/decode; SGLang GDN backends (Triton / FlashInfer); causal_conv1d + conv1d metadata work in vLLM GDN attn

Ask for InferRSI

Own this as an RSI track: distill kernel decision cases, run/guide the evidence loop on L20X, and feed verified recipes back to system1-omni model tips.