Staff Backend Engineer - K8 (Envoy)

Overview

Staff Backend Engineer responsible for designing and delivering scalable infrastructure for AI inference workloads. The role provides technical direction across teams and focuses on reliable, secure, high-performance platform services.

What you'll do

• Architect and build an AI Inference Gateway platform for machine learning and generative AI workloads.
• Design request routing, endpoint abstraction, traffic shaping, load balancing, failover, caching, and policy enforcement.
• Develop infrastructure components in Go, Java, or Python.
• Build multi-tenant capabilities for authentication, authorization, quotas, cost attribution, rate limiting, and governance.
• Lead architecture reviews, raise engineering standards, and mentor senior engineers.
• Define observability standards and resolve complex production issues through root-cause analysis.
• Optimize latency, throughput, resiliency, and infrastructure efficiency.

What you'll need

• 8+ years of professional software development experience.
• 5+ years designing and operating large-scale distributed systems in production.
• Strong programming expertise in Go, Java, or Python.
• Experience building highly available, mission-critical platform or infrastructure services.
• Experience with API platforms, service gateways, service mesh, or large-scale networking infrastructure.
• Experience with Kubernetes, containers, and a major cloud platform.
• Strong understanding of observability, reliability engineering, capacity planning, and production operations.

Nice to have

• Experience with AI/ML inference platforms, LLM gateways, model serving, or GPU workloads.
• Expertise in Gateway API, ingress controllers, service mesh, or platform networking.
• Experience with inference frameworks such as vLLM, Triton, TensorRT-LLM, Ray Serve, KServe, or SGLang.
• Experience with Kafka, Cassandra, Redis, MongoDB, or similar distributed data systems.

Details

• Location: Bengaluru.
• Work model: Hybrid, onsite, or remote, with hybrid employees expected to work at least 3 days per week in the office.