Senior Staff QA

Overview

Lead the end-to-end quality strategy for JazzX AI's AI-powered platforms, agent workflows, LLM outputs, APIs and cloud services while managing a QA team.

What you'll do

• Build and own the end-to-end QA strategy across AI copilots, workflows and enterprise deployments.
• Lead, mentor and scale a team of QA engineers and automation specialists.
• Partner with Product, ML Engineering and Platform Engineering to embed quality in early design phases.
• Define and enforce quality gates and release-readiness standards.
• Develop evaluation systems for LLM accuracy, hallucination detection, factuality, safety and bias mitigation.
• Design scalable tests for agent reasoning, tool-use correctness and multi-step workflow validation.
• Create synthetic datasets and scenario-based tests for edge-case prediction and regression testing.
• Validate model performance across fine-tuning, RAG, knowledge-grounding and inference scenarios.
• Architect and manage automation frameworks using Python/Java/JS, Playwright, Selenium, PyTest and Postman.
• Integrate automated suites into CI/CD pipelines (GitHub Actions/GitLab/Jenkins).
• Automate API, microservices and backend testing for cloud architecture running on AWS/GCP.
• Perform load, stress, spike and endurance/soak testing using tools such as JMeter, LoadRunner, Gatling, k6 or Locust.
• Own the release sign-off process for all AI software components and platform features.
• Conduct RCA, maintain issue dashboards and drive continuous improvement in the QA lifecycle.
• Collaborate with DevOps/ML Ops to monitor model drift, performance anomalies and post-production behaviors.
• Prepare QA documents including strategies, test plans, evaluation reports and execution metrics.
• Communicate risks, quality insights and impact analysis clearly to leadership.

What you'll need

• 12+ years of experience in software QA.
• Strong hands-on experience with test automation frameworks and scripting in Python/Java/JavaScript.
• Experience working with microservice-based architecture.
• Working knowledge of AI/ML systems, including datasets, model evaluation, vector stores, RAG and LLM operations.
• Automated API, microservices and backend testing for cloud architecture running on Azure/AWS.
• Experience testing distributed cloud systems, APIs and high-performance backend services.
• Strong analytical and problem-solving skills with the ability to drive organization-wide quality initiatives.

Nice to have

• Experience testing LLMs, RAG pipelines or autonomous AI agents.
• Familiarity with AI observability tools and logging/monitoring stacks (ELK, Prometheus, Datadog).
• Exposure to security testing, performance/load testing and compliance (SOC2, GDPR).
• Ability to operate in a fast-paced, zero-to-one startup environment.

Details

• Location: Bengaluru, Karnataka.