Senior Technical Product Manager - AI Agents, Evals & Reliability

Overview

Lead the strategy, system design, evaluation, and reliability of AI-agent products. Partner closely with engineering and ML teams to turn model capabilities into predictable, trustworthy user experiences.

What you'll do

• Define end-to-end AI system requirements from capability through user impact.
• Translate model capabilities, data constraints, and evaluation results into product decisions.
• Balance quality, latency, cost, reliability, and user experience.
• Partner with ML, backend, and mobile engineers on system design and iteration.
• Build evaluation frameworks using offline metrics, online experiments, and human feedback.
• Drive execution through clear specifications and prioritization.
• Own product quality, including correctness, predictability, and user trust.

What you'll need

• Strong computer science fundamentals in algorithms, data structures, and system design.
• Solid understanding of machine learning fundamentals and production AI systems.
• Experience with AI-powered products, including LLM-based systems.
• Experience with model evaluation, prompt or pipeline iteration, and feedback loops.
• Understanding of model limitations, hallucinations, bias, and drift.
• Significant experience owning complex technical products end-to-end.
• Ability to work closely with senior engineers and ML teams in ambiguous environments.

Nice to have

• Experience shipping AI-heavy consumer products.
• Engineering background or highly technical product management experience.
• Experience defining evaluation metrics for ML systems.
• Experience with AI UX patterns and failure handling.
• Experience in zero-to-one product environments.

Details

• Location: China.