AI Automation Engineer - Test Generation & Agent Systems (Sr Staff)

Overview

Build AI automation for a QE (quality engineering) platform, where Jira/product specs are turned into first-draft Playwright tests and an agent maintains the live test suite.

What you'll do

• Design and build a Claude-powered test generation pipeline from Jira tickets or product specifications.
• Build a test maintenance agent that monitors tests, detects flakiness/failures, diagnoses causes, and proposes or auto-applies fixes with human-in-the-loop review.
• Maintain the prompt engineering layer, including a prompt library and an evaluation harness to score generated test quality.
• Define an acceptance workflow for AI-generated tests, including which require engineer review before merge vs auto-accept.
• Instrument the agent layer with metrics like cost per generated test, acceptance rate, time-to-fix, and overall flakiness reduction.
• Collaborate with automation engineers to improve generation quality for complex business logic.
• Stay current with LLM tooling and evaluate new approaches to improve speed or quality.

What you'll need

• Production experience with Anthropic Claude API or OpenAI API, including tool use, multi-step agent patterns, and structured output.
• 8+ years of strong TypeScript or Python experience.
• Ability to evaluate LLM output quality using rigorous evals.
• Experience designing agentic systems with graceful failure handling (retries, fallbacks, and human escalation paths).
• Comfort with ambiguity and incomplete requirements.

Nice to have

• Playwright (or comparable E2E testing framework) experience to read and reason about test code fluently.
• Experience with test infrastructure such as fixture factories, seed scripts, and environment isolation.
• Prior work building agents with persistent memory, multi-tool orchestration, or complex retry logic.

Details

• Location: Bangalore.