Senior LLM Systems Engineer

Responsibilities

• Own and improve LLM-driven components of the oracle automation stack
• Improve LLM accuracy through better prompts, model selection, and retrieval
• Optimize system performance regarding latency, token usage, and cost
• Design resilience mechanisms including validation, retries, and fallbacks
• Build evaluation datasets, regression tests, and observability dashboards
• Improve agent orchestration and tool use across internal services and APIs
• Debug live production issues and improve operational runbooks

Requirements

• 3+ years of professional software engineering experience in Python or TypeScript
• Hands-on experience building production systems using LLMs, agents, or RAG
• Experience designing evaluations, test datasets, or quality metrics for AI systems
• Strong debugging skills across APIs, databases, queues, and logs
• Practical understanding of prompt engineering and structured output validation
• High agency and strong written communication skills

Preferred Qualifications

• Experience with oracle systems, prediction markets, or DeFi protocols
• Knowledge of UMA, optimistic oracle mechanisms, or Polymarket
• Experience building agentic systems with browser automation or tool use
• Familiarity with LLM tracing and AI observability tools
• Experience with Postgres, data pipelines, and event-driven architectures
• Proficiency with GCP, Cloud Run, GitHub Actions, or Terraform

About the Company

Risk Labs is the core team behind UMA and Across, building infrastructure that pushes crypto forward. We focus on production AI systems that directly impact live protocol operations.