Staff AI Engineer

Responsibilities

• Set technical direction for agent systems across teams, including orchestration, tool use and memory management
• Build and operate evaluation frameworks, agent observability, execution controls and model routing
• Establish evaluation practices measuring task completion, accuracy, security and cost per task
• Make agent workflows resilient to tool failures, timeouts and rate limits
• Work with security and platform teams on scoped permissions, data isolation and audit trails
• Lead model selection and cost optimization against real workloads
• Help teams use AI coding agents effectively and measure impact
• Mentor engineers and technical leads through reviews, pairing and written guidance

Requirements

• 8-10+ years of software engineering experience with technical leadership
• Hands-on experience building and operating LLM-based agents in production
• Track record improving agent reliability through evaluations and regression testing
• Strong knowledge of agent architectures, tool use and orchestration
• Experience with managed cloud model platforms or direct provider APIs
• Strong backend and distributed systems fundamentals
• Experience with production observability and safe rollouts
• Understanding of security risks in agent systems such as prompt injection

Preferred Qualifications

• Experience with GCP, Go or Kubernetes
• Experience building sandboxed environments for agents
• Offensive security, application security or security tooling experience
• Experience introducing AI coding tools across engineering teams

Benefits

• Competitive compensation and equity plan
• 401(k) (US) or pension (EU)
• Medical, dental, vision and life insurance (US) or statutory healthcare (EU)
• Wellness, home office and learning stipends
• Flexible paid time off and paid parental leave

About the Company

Cobalt runs AI agents in production that test real customer applications. It operates in the pentesting industry.