Senior Quality Test Engineer II

Overview

Drive quality engineering and test design for AI-driven and agentic solutions built on Salesforce and integrated systems. Own quality strategy for deterministic Salesforce functionality and non-deterministic AI capabilities, while mentoring junior QA engineers and representing quality and responsible-AI considerations throughout the SDLC.

What you'll do

• Interface with developers, business analysts, and product owners to finalize testable requirements and acceptance criteria for deterministic and AI-driven features.
• Write and review detailed test plans, test cases, and evaluation frameworks for complex system components, including AI agents, automations, and generative AI features.
• Design and maintain evaluation suites, including golden datasets, regression benchmarks, and output-quality rubrics, for agentic and generative AI capabilities.
• Execute and automate functional, regression, integration, and performance testing across Salesforce and adjacent systems.
• Identify, document, and drive resolution of complex defects, including hallucination, drift, bias, and inconsistent AI outputs.
• Partner with developers to shift quality left by reviewing designs, prompts, and agent configurations before build.
• Establish and evangelize quality and responsible-AI standards, including guardrails for validating AI-generated code and outputs before production release.
• Support release management, packaging, and compliance/audit requirements with appropriate test evidence and traceability.
• Monitor production AI features for quality drift and performance degradation, and define escalation paths when thresholds are breached.
• Train and mentor less-senior QA engineers on Salesforce testing methodologies, automation frameworks, and AI/agentic testing practices.
• Keep abreast of Salesforce platform releases, Agentforce/AI feature developments, and emerging AI quality-assurance tools and techniques.
• Perform all other duties as assigned.

What you'll need

• 5+ years of Software Quality Engineering / Test Engineering experience, with 2+ years testing enterprise Salesforce implementations.
• BS Engineering/Computer Science or equivalent experience.
• Advanced knowledge of software quality methodologies, including Agile/Scrum, risk-based testing, and shift-left testing, as applied to Salesforce release cycles and org governance.
• Strong proficiency in Salesforce test automation, including Apex unit/integration testing, LWC Jest testing, and UI automation frameworks such as Provar, Selenium, or Playwright.
• Strong understanding of the Salesforce data model and sharing/security model, including how data quality and permission structures affect functional and AI feature testing.
• Working knowledge of SOQL/SOSL for test data validation and querying, and integration testing across REST/SOAP APIs, Platform Events, and middleware such as MuleSoft.
• Hands-on experience testing AI-driven and agentic features, including evaluation rubrics, golden datasets, regression suites for prompts and agent behavior, and acceptable-variance thresholds.
• Experience testing Agentforce agents, Copilot/Einstein-style features, and prompt-builder workflows for accuracy, grounding, hallucination risk, and adherence to business rules.
• Experience evaluating AI outputs for bias, data privacy, and responsible-AI concerns, and partnering with stakeholders to define acceptance criteria for “good enough” AI behavior.
• Experience using AI-assisted testing tools, such as AI-driven test case generation, self-healing test automation, and AI code/PR review assistants.
• Experience monitoring AI feature performance and drift in production and defining feedback loops for continuous evaluation.
• Strong knowledge of test-driven and behavior-driven development practices, and ability to shape acceptance criteria before code is written.
• Ability to write and review detailed test plans, test cases, and evaluation frameworks for complex system components, including AI agent/automation designs.
• Ability to triage and drive resolution of complex defects across Apex, LWC, Flow, integrations, and AI/agentic behavior.
• Strong research skills, including staying current on Salesforce release notes, Agentforce/AI feature rollouts, and emerging AI quality-assurance practices.
• Knowledge of the broader sales technology tool stack, including CPQ/Incentive Comp platforms, Gainsight, and Monday.com, and how quality risk surfaces across integrated systems.
• Ability to interface competently with developers, business analysts, product owners, and platform admins to finalize testable requirements.
• Good oral and written communication skills, including the ability to explain AI evaluation results, confidence levels, and residual risk to non-technical stakeholders.

Nice to have

• Experience testing AI/ML-driven, LLM-powered, or generative AI features.
• Salesforce certifications, including Platform App Builder and/or AI Associate/AI Specialist.
• Experience in regulated, audited, or compliance-driven release environments.

Details

• Location: Chennai.
• The role offers flexible hours, allowing employees to flex the times they work during the day.