Member of Technical Staff - Inference

About the Company

Prime Intellect is building the open superintelligence stack: the infrastructure frontier AI labs build internally, made available to every ambitious AI team. Our platform, Lab, unifies compute, environments, evaluations, secure sandboxes, high-performance training, and deployment into one full-stack system for post-training at frontier scale.

Responsibilities

• Build a multi-tenant LLM serving platform operating across cloud GPU fleets
• Design placement and scheduling algorithms for heterogeneous accelerators
• Implement multi-region/zone failover and traffic shifting for resilience
• Integrate and contribute to LLM inference frameworks like vLLM, SGLang, and TensorRT-LLM
• Optimize configurations for tensor, pipeline, and expert parallelism
• Profile kernels, memory bandwidth, and transport to apply quantization and speculative decoding
• Establish CI/CD with artifact promotion and performance gates
• Build observability metrics, logs, and tracing for SLO management

Requirements

• 3+ years building and running large-scale ML/LLM services with latency/availability SLOs
• Hands-on experience with vLLM, SGLang, or TensorRT-LLM
• Familiarity with distributed serving infrastructure such as NVIDIA Dynamo
• Deep understanding of prefill vs. decode, KV-cache behavior, batching, and sampling
• Proficiency in Python and PyTorch
• Experience with Kubernetes and cloud deployment patterns (AWS/GCP)
• Knowledge of GPU architecture, CUDA runtime, NCCL, and InfiniBand

Preferred Qualifications

• Kernel-level optimization experience with CUDA/Triton
• Proficiency in Rust or C++
• Experience with Kafka, Redis, gRPC, or Prometheus/Grafana
• Infrastructure-as-code experience with Terraform or Ansible
• Contributions to open-source serving, inference, or RL infrastructure projects

Benefits

• Cash compensation of $150,000 - $300,000 with significant equity
• Flexible work arrangement (remote or San Francisco office)
• Full visa sponsorship and relocation support
• Professional development budget
• Regular team off-sites and conference attendance