Machine Learning Engineer

Responsibilities

• Optimize inference latency, throughput, and cost for large-scale ML models in production
• Profile and bottleneck GPU/CPU inference pipelines including memory, kernels, batching, and IO
• Implement and tune techniques such as quantization, KV-cache optimization, speculative decoding, and model pruning
• Collaborate with research engineers to productionize new model architectures
• Build and maintain inference-serving systems using Triton, custom runtimes, or bespoke stacks
• Benchmark performance across NVIDIA/AMD GPUs, CPUs, and cloud setups
• Improve system reliability, observability, and cost efficiency under real workloads

Requirements

• Strong experience in ML inference optimization or high-performance ML systems
• Solid understanding of deep learning internals such as attention, memory layout, and compute graphs
• Hands-on experience with PyTorch or similar frameworks and model deployment
• Familiarity with GPU performance tuning including CUDA, ROCm, Triton, or kernel-level optimizations
• Experience scaling inference for real users
• Ability to work in fast-moving startup environments

Preferred Qualifications

• Experience with LLM or long-context model inference
• Knowledge of inference frameworks like TensorRT, ONNX Runtime, vLLM, or Triton
• Experience optimizing across different hardware vendors
• Open-source contributions in ML systems or inference tooling
• Background in distributed systems or low-latency services

About the Company

We are a Series A startup focused on building high-performance, reliable, and cost-efficient machine learning systems that serve real users.