Machine Learning Engineer

Overview

On-site AI Evaluation Engineer role to build evaluation scaffolding and quality layers for an AI product in production.

What you'll do

• Build and maintain evaluation harnesses for AI-facing features to measure and tune system quality.
• Own end-to-end and API test infrastructure to support a continuous, daily-release cycle.
• Design and execute host-behavior probes using scripted user sessions across AI assistants.
• Gate production releases using UAT, regression analysis, and quality reporting.
• Monitor product behavior across AI assistants while accounting for differences by host and changes over time.

What you'll need

• Background in SDET/QA automation or ML evaluation with ownership of test/evaluation infrastructure.
• Programming skills in Python or TypeScript.
• Experience in API-level testing.
• Experience with tools like Playwright or Cypress.
• Familiarity with LLM applications or an interest in evaluating non-deterministic systems.
• Ability to define workflows independently for evaluation and verification.

Details

• Location: HSR, Bengaluru (On-site).