Specialist 3 – AI-Testing, Advisory

Overview

Specialist 3 role in PwC's Data, Analytics & AI practice focused on AI testing and quality assurance, designing and building automated validation for AI applications and models.

What you'll do

• Define and implement enterprise-wide AI testing and quality assurance strategies.
• Drive continuous improvement in test automation, quality metrics, and release processes.
• Build and maintain scalable automation frameworks for AI applications and APIs.
• Develop automated validation pipelines for machine learning models.
• Integrate automated testing into CI/CD workflows.
• Validate LLM responses using automated evaluation frameworks.
• Design testing strategies for chatbots, copilots, agents, and RAG applications.
• Evaluate prompt effectiveness and response quality.
• Implement automated prompt testing and output validation.
• Conduct load, stress, scalability, and resilience testing for AI applications.

What you'll need

• Bachelor of Engineering (B.Tech / M.Tech / MBA / MCA education qualification listed in the posting).
• 0-2 years of experience.
• Playwright.
• Selenium.
• MLflow.
• OpenAI Evaluations.
• CI/CD Pipelines.
• Azure Devops (required skill).

Nice to have

• Experience testing LLMs, RAG architectures, AI Agents, and Copilot.

Details

• Location: Bengaluru Millenia.
• Visa sponsorship: No.
• Government clearance required: No.
• Job posting end date: May 31, 2026.