AI Platform Engineer II

Overview

Own significant areas of Enterprise AI's platform, ensuring reliability, security, performance and cost efficiency of AI services, and make AI consumable at enterprise scale through gateways, model endpoints and context services.

What you'll do

• Own significant platform areas with accountability for reliability, security, performance and cost efficiency of AI services.
• Operate the gateways, model endpoints and context services through which every team reaches models, tools and data.
• Build automation, self-service and reusable components that reduce manual operations work across the team.
• Embed AI security, data protection and responsible AI controls into the platform layer.
• Mentor junior and mid-level platform engineers, building team capability over time.
• Contribute to vendor and standards conversations that shape the platform's direction.

What you'll need

• At least 6 years of experience running SaaS, cloud or platform-engineering services in a production setting.
• At least 2 years operating AI/ML or GenAI workloads or services.
• Strong proficiency in Python; working proficiency in at least one additional language (for example, Go, TypeScript or Java).
• Hands-on experience with Kubernetes and container platforms, Infrastructure-as-Code (Terraform), GitOps and modern CI/CD.
• Hands-on experience operating API gateways or AI/LLM gateways, including routing, rate limiting, authentication and policy enforcement for AI traffic.
• Working knowledge of Model Context Protocol (MCP) and agentic patterns, including tool calling, orchestration and agent frameworks, and of how context is retrieved, managed and governed for LLM applications.
• Experience operating production services with defined SLOs, observability and incident management.
• Daily use of AI coding agents (Claude Code, GitHub Copilot, Databricks AI Assistant) as a primary development mode.
• Solid understanding of identity and access management, secrets management and data protection for cloud and AI services.
• Bachelor's degree in Computer Science, Software Engineering, or equivalent practical experience.
• Required skills: Kubernetes, containers, Terraform, GitOps, CI/CD, DevSecOps, platform reliability engineering, and at least one major cloud (Azure preferred) with multi-cloud and hybrid architecture awareness.
• Required skills: API gateway patterns, AI/LLM gateway architecture and operations, traffic management and policy enforcement.
• Required skills: LLM deployment, versioning and monitoring; prompt and evaluation workflows; RAG architectures; vector search fundamentals.
• Required skills: Model Context Protocol, tool calling, workflow orchestration, agent orchestration frameworks and context engineering.
• Required skills: metrics, logging, tracing and alerting (for example, OpenTelemetry, Prometheus, Grafana or cloud-native equivalents) applied to AI services.
• Required skills: IAM, OAuth/OIDC, secrets management, network security and data privacy controls for AI workloads.

Nice to have

• Master's degree in Computer Science, Distributed Systems or SRE.
• Azure AI services (Azure OpenAI/AI Foundry, Azure API Management), with AWS and GCP AI services (for example, Bedrock, Google Cloud AI platform) desired.
• Model serving and GPU infrastructure: inference runtimes (for example, vLLM, KServe, Triton), GPU scheduling and autoscaling.
• Vector and retrieval stacks: vector databases and hybrid search (for example, pgvector, Azure AI Search, OpenSearch or dedicated vector stores).
• Agent and LLM frameworks: LangChain/LangGraph, Microsoft Agent Framework or comparable; agent interoperability protocols; evaluation tooling.
• Gateway technologies: Kong AI Gateway or comparable (for example, Envoy-based gateways, LiteLLM, Portkey, Azure API Management).
• Platform tooling: Databricks, policy-as-code (OPA/Kyverno), service mesh, internal developer platforms (for example, Backstage) and FinOps for AI (token economics and unit-cost reporting).
• AI security and resilience: guardrail frameworks, AI red-teaming and resilience testing of AI services.
• Bicep experience.

Details

• Location: Bengaluru, India.