Post-Training Efficiency: Parameterisation × Alignment × Quantisation

Given a fixed memory budget, which post-training recipe should you actually pick?

Method choices in post-training are usually justified by numbers measured under different bases, different data and different eval harnesses — which makes them uncomparable. This project fixes all three and varies one axis at a time, at two model scales, and reports the efficiency side (MFU, peak memory, throughput) alongside the quality side.

Status: in development. The matrix below is fixed; results are not in yet. No numbers are reported until the runs are complete.

---

Why two scales

A recipe that wins at 0.8B does not necessarily win at 9B. Low-rank adaptation is known to close the gap with full fine-tuning as models grow, but the shape of that convergence is rarely measured under controlled conditions. So the matrix runs twice:

Tier	Model	Hardware	Purpose
Breadth	Qwen3.5-0.8B	1 GPU	Full 12-cell matrix — cheap, fast, many ablations
Scale	Qwen3.5-9B	4×A100 40GB, DeepSpeed ZeRO-3	Key cells only — does the ranking hold an order of magnitude up?

The deliverable is a scale-dependent decision table, not a single ranking.

Matrix

Axis	Values
Parameterisation	full / LoRA (r=8) / DoRA (r=8) / QLoRA (4-bit)
Alignment	SFT-only / DPO / GRPO
Post-training quantisation	none / AWQ 4-bit
Quality eval	GSM8K, SVAMP, MMLU, format compliance, output length
Efficiency eval	MFU, peak memory (decomposed), tokens/s, scaling efficiency

Base model, dataset and eval harness are held fixed across every cell.

Distributed setup

Full-parameter 9B does not fit on 4×40GB without offload. The arithmetic:

Component	Size
bf16 weights	18 GB
bf16 gradients	18 GB
AdamW fp32 states + fp32 master (9B × 12 B)	108 GB
Total	144 GB

Sharded across 4 GPUs that is 36 GB/card before activations — an OOM on a 40 GB card. Moving the optimiser states to host memory (Delta nodes carry 256 GB) leaves (18+18)/4 = 9 GB/card plus activations, which fits.

The cost of that move is measured rather than assumed: configs/ds_zero3_offload.json and configs/ds_zero3_nooffload.json are identical except for the offload target, and both run at ≤2B where either fits. The delta is the throughput price of offloading.

Failure modes worth knowing

stage3_gather_16bit_weights_on_model_save temporarily materialises the full 18 GB of weights on rank 0. A run can train happily for hours and then die at the first save_steps — the most expensive way to discover a memory problem. Verify a checkpoint write inside the smoke test, not at hour three.

Roadmap

• DeepSpeed ZeRO-3 configs (offload / no-offload pair)
• Delta reconnaissance script
• Multi-GPU SFT submission script
• src/train_sft.py — training entry point
• src/train_dpo.py, src/train_grpo.py
• Evaluation harness (single command, JSON output)
• Smoke test, including a checkpoint write
• Breadth tier: 12-cell matrix at 0.8B
• Scale tier: key cells at 9B
• AWQ quantisation and throughput measurement
• Decision table

Repository layout

configs/    DeepSpeed configs, experiment definitions
scripts/    cluster reconnaissance and submission
src/        training and evaluation entry points

Cluster notes

Written against NCSA Delta, whose constraints shape the submission script:

• 2-day wall clock → checkpoint/resume plus --dependency=afterany job chains
• 16 CPU cores per GPU → --cpus-per-task=64 for a 4-GPU job
• Accounts are partition-scoped; the account name is passed at submit time, never hard-coded

Run scripts/delta_recon.sh first — it reports account names, balances, partition access and whether compute nodes reach the network, which determines whether model weights must be cached offline in advance.

Model and compute

• Base models: Qwen3.5-0.8B, Qwen3.5-9B (Apache-2.0)
• Compute: NSF ACCESS allocation on NCSA Delta

License

MIT — see LICENSE.