Engineering Manager, Agent Oversight

Overview

Engineering Manager for Agent Oversight, leading a team building a platform to monitor, evaluate, and improve agentic applications for enterprise and government customers.

What you'll do

• Lead a multi-disciplinary team of software and ML engineers to drive technical delivery.
• Own the platform roadmap for deployment, monitoring, evaluation, and ML-driven improvement.
• Translate enterprise and government requirements into platform capabilities via cross-functional work.
• Build and ship features end-to-end, from system design through debugging and testing.
• Drive experimentation to validate and improve platform capabilities using real customer usage.
• Set technical direction, culture, and processes for a fast-growing team.
• Mentor and develop engineers and ML engineers/scientists as the team scales.

What you'll need

• 7+ years of engineering experience.
• 2+ years directly managing engineers or ML engineers for a production ML/LLM-powered system.
• Hands-on familiarity with agent architectures including tool use, planning, and multi-agent orchestration.
• Ability to review ML experiment design or evaluation methodology and ask sharp questions.
• Track record owning the full lifecycle of platform-level infrastructure from design through scaling.
• Experience collaborating with product managers, forward deployed engineering teams, and customers.
• Track record building and growing high-performing engineering teams, including hiring and retention.

Nice to have

• Experience building or overseeing evaluation, monitoring, or observability systems for production ML/LLM products.
• Strong grasp of the ML/agent development lifecycle from experimentation through production deployment.
• Deep understanding of modern LLMs and agentic system design, including prompt- and system-level optimization.
• Published research, open-source contributions, or patents in agentic systems, LLMs, or applied ML.
• Ability to operate in ambiguous problem spaces by balancing research and pragmatic product constraints.

Details

• US-based team with members across New York and San Francisco.