Machine Learning Engineer

Overview

The Splunk AI Platform and Services team builds and operates the core runtime and developer infrastructure powering AI across Splunk and Cisco, sitting at the intersection of Site Reliability Engineering, Distributed Systems, and ML Infrastructure.

What you'll do

• Operate and optimize multi-tenant LLM inference, model serving, VectorDB/RAG, and MCP platforms across global cloud and GPU clusters.
• Establish and monitor SLOs, SLIs, automated health checks, and observability dashboards (Prometheus, Grafana, Splunk, OpenTelemetry).
• Build automation for seamless rollouts, autoscaling, canary deployments, self-healing, and disaster recovery.
• Tune throughput, latency, model scheduling, batching, caching, and resource allocation using Kubernetes, vLLM, and Ray.
• Lead complex production incident response, conduct 5-Why root-cause analyses, and implement systemic remediations to eliminate repeat outages.
• Partner with AI researchers, product developers, and platform teams on production readiness reviews (PRRs) and participate in a sustainable on-call rotation.

What you'll need

• Bachelor's degree in CS/Engineering with 5+ years of relevant experience, or Master's with 3+ years.
• 3+ years of programming experience in Python, Go, or Java building automation, tooling, or production backend services.
• Solid experience running distributed, containerized systems in production using Kubernetes, microservices, and CI/CD.
• Proven background in production operations, distributed troubleshooting, and monitoring stacks (Prometheus, Grafana, Splunk, or OpenTelemetry).
• Hands-on working knowledge of AWS, Azure, or GCP.

Nice to have

• Hands-on experience with vLLM, TensorRT-LLM, Ray Serve, or KServe.
• Experience managing GPU scheduling, capacity planning, model parallelism, and performance profiling.
• Experience operating Vector databases, search engines, or distributed cache layers (e.g., Milvus, Pinecone, Redis).
• Familiarity with RAG pipelines, model routing, Model Context Protocol (MCP), and agent runtimes.
• Track record designing multi-region high-availability systems, automated remediation, and chaos engineering practices.

Details

• Location: Hyderabad, India.