Senior ML Engineer

About the Company

Nebius is leading a new era in cloud infrastructure for the global AI economy. We are building a full-stack AI cloud platform that supports developers and enterprises from data and model training through to production deployment, without the cost and complexity of building large in-house AI/ML infrastructure.

Responsibilities

• Identify LLM inference bottlenecks to drive production speedups
• Squeeze maximum performance for a wide range of LLM architectures at scale
• Implement novel speculative decoding architectures
• Optimise components of various LLM designs (dense/MoE, autoregressive/parallel)
• Design and productionise low-precision (FP8, NVFP4/MXFP4) training and inference pipelines

Requirements

• Profound understanding of machine learning theoretical foundations and transformer architecture
• Experience profiling GPU workloads using Nsight, PyTorch profiler, or similar tools
• Understanding of GPU memory hierarchy and compute/memory tradeoffs
• Familiarity with LLM concepts like MHA, RoPE, KV-cache, Flash Attention, and quantisation
• Understanding of large neural network training performance (sharding, custom kernels, hardware features)
• Strong software engineering skills in Python
• Deep experience with modern deep learning frameworks
• Proficiency in CI/CD, version control, and unit testing
• Strong communication and leadership abilities

Preferred Qualifications

• Experience with open-source inference engines like vLLM, SGLang, or TensorRT-LLM
• Experience with kernel languages or DSLs such as Triton, Cute, CUTLASS, or CUDA
• Track record of delivering products in dynamic startup-like environments
• Experience developing large distributed systems or high-load web services
• Contributions to open-source projects