Overview
Staff Machine Learning Engineer to shape the architecture and strategy of Zendesk’s GenAI platform, leading cross-product LLM evaluation, safety standards, and platform reliability.
What you'll do
- Architect and lead delivery of cross-product GenAI platform capabilities (LLM Proxy, model registry integrations, vendor abstraction, cost/usage attribution).
- Design and scale evaluation and benchmarking frameworks (A/B, offline, continuous regression tests) to gate model releases.
- Define company-wide standards for safety, tone, and reasoning evaluation and drive adoption of rubrics and automated checks.
- Identify systemic failure modes and prioritize mitigations, monitoring, and retraining strategies with ML teams.
- Improve platform reliability, observability, and capacity planning for LLM services, including rate limiting, throttling, and SLA practices.
- Enable agentic workflows and safe tool use by defining integration patterns and security boundaries.
- Mentor senior engineers and coordinate cross-team roadmaps.
What you'll need
- 8+ years building distributed systems and ML infrastructure with production delivery across cross-teams.
- Deep understanding of LLMs, inference serving patterns, vendor routing strategies, and ML workload platform design.
- Strong system design skills including scalable architectures, reliability engineering, capacity planning, and cost optimization.
- Proficiency in Python (or comparable server-side language), Kubernetes, cloud infrastructure, and observability tooling.
- Experience creating evaluation frameworks, gold-standard datasets, and regression suites for language models.
- Excellent stakeholder skills to synthesize product, research, and engineering constraints into platform solutions.
- BS in Computer Science, Engineering, or related field, or equivalent practical experience.
Details
- Location: Pune, India.
- Work mode: Hybrid; local office attendance part of the week with remote flexibility for part of the week (team-level schedule).
Read the full description and apply on the company’s own careers page.
