Sr. AI Engineer 4B

Overview

Build and execute automated tests for traditional application layers and AI-powered components, including LLM responses, RAG pipelines, and agentic workflows. This is a hands-on individual contributor role within an Advanced AI Practice.

What you'll do

• Build and maintain automated functional, integration, and regression test scripts.
• Design and execute REST API tests for application and AI/model endpoints.
• Validate LLM responses, consistency, hallucinations, and incorrect outputs.
• Extend end-to-end automation suites across UI, API, and data layers.
• Integrate automated tests into Jenkins, GitHub Actions, or Azure DevOps pipelines.
• Log, track, and triage defects using Jira or HP ALM.
• Prepare test cases, test data, execution reports, and coverage metrics.

What you'll need

• 3+ years of hands-on QA or test automation engineering experience, especially in AI application development.
• Strong Java or Python programming skills.
• Experience with Selenium, Cypress, JMeter, or Playwright.
• Experience testing REST APIs using Postman or RestAssured.
• Experience with functional, integration, and regression testing.
• Working knowledge of Git/GitHub and CI/CD pipelines.
• Bachelor's degree in Computer Science, Information Technology, Electronics, or a related field.

Nice to have

• Exposure to AI/LLM evaluation tools such as Ragas, DeepEval, Promptfoo, LangSmith, or Langfuse.
• Experience testing RAG pipelines, agentic workflows, or conversational AI applications.
• Performance or load testing experience using JMeter.
• Exposure to Docker, Kubernetes, GCP, or Agile/Scrum.
• ISTQB, cloud, or Agile/Scrum certifications.

Details

• Location: Bangalore, Karnataka, India.
• Work mode: Hybrid.
• Employment type: Regular.
• Work shift: Day shift.