Hands-on projects across the modern AI engineering stack: LLM evaluation, RAG, agents, fine-tuning, inference serving, GPU performance, observability and distributed training. Built during the Nebius Academy AI Performance Engineering program, each project is a working system with measured results and an honest write-up of what did and didn't work.
| Project | What it is | Stack |
|---|---|---|
| text2sql-observability | Text-to-SQL service tuned from a P95 of 84 s to 4.4 s at 11.8 RPS on 1× H100, with no accuracy loss | vLLM · LangGraph · Langfuse · Prometheus/Grafana |
| gpu-perf-lab | Roofline analysis of real kernels, a 2.9× faster LLM decode loop, launch overhead and CUDA graphs | PyTorch profiler · torch.compile · CUDA graphs |
| data-analyst-agent | Conversational data-analyst agent: LLM router + ReAct over typed tools, persistent memory, 95% routing accuracy | LangGraph · Pydantic · MCP · Streamlit |
| fin-rag | RAG over SEC filings (FinanceBench) evaluated on correctness, faithfulness and hit-rate, with improvement cycles | LangChain · FAISS · RAGAS |
| llm-copywriter-eval | LLM content generation with a human rubric, LLM-as-a-judge, and judge–human agreement analysis | OpenAI SDK · Pydantic · Nebius |
| skypilot-ddp-training | Multi-node PyTorch DDP on Kubernetes H100s, with a 1→4 GPU NCCL communication-overhead analysis | PyTorch DDP · SkyPilot · mk8s |
| qlora-attention-lab | QLoRA fine-tuning of small LLMs to answer at child/student/expert level, plus attention implemented from scratch | PEFT · bitsandbytes · TRL · PyTorch |
| seq-models-nlp | Subword tokenization → Word2Vec → MLP news classifier (91%), and a char-level LSTM language model | PyTorch · gensim · 🤗 tokenizers |
| multimodal-vision-lab | Image captioning, visual QA and CLIP-based face recognition, with failure analysis | BLIP · CLIP · 🤗 Transformers |
| torch-optim-lab | SGD, Momentum, AdaGrad and Adam from scratch, plus numerical stability and L1 sparsity | PyTorch |
- Nathanael Bloch: @nbloch
- Tomer Porat: @analytics972