Overview
Specialist, Site Reliability Engineering role based in Hyderabad, responsible for keeping the inference platform running reliably by building monitoring, alerting, capacity dashboards, incident handling and automation to support production models.
What you'll do
- Build observability for the inference layer, from GPU to gateway, including GPU health and utilisation (DCGM, including XID and ECC errors) and engine metrics such as time to first token, inter-token latency, queue depth, KV cache use, preemptions, error rates and token throughput.
- Run synthetic end-to-end probes that check the content of model responses, not only the HTTP status.
- Create the capacity and status dashboards that show what runs where and how much capacity is free.
- Implement the SLOs, error budgets and alerts that the lead defines; run on-call and incident response; facilitate post-incident reviews.
- Handle incidents, problems and changes through IT service management (ServiceNow or similar).
- Automate operations: health checks, self-healing, safe upgrades, and the signals that tell the gateway when to spill traffic over to cloud models.
- Coordinate node draining and maintenance with the hardware and cluster team.
- Run capacity and resilience tests and keep runbooks and the support model up to date.
- Work with the hardware and cluster team on hardware faults and cluster issues, and with security on access and logging.
What you'll need
- Bachelor's degree in information technology, Computer Science or related field.
- 3 to 6 years of hands-on experience working in full-stack development.
- DevOps or platform engineer running production services.
- Hands-on Prometheus and Grafana (or equivalent), alerting and on-call practices.
- Kubernetes and Linux operations; scripting in Python or Bash.
- Familiarity with LLM serving engines (vLLM, SGLang or Triton) and their metrics.
- Infrastructure as code and CI/CD.
Nice to have
- Experience operating GPU clusters (NVIDIA DCGM, GPU Operator) or HPC environments.
- Experience with ServiceNow incident, problem and change processes.
- Cybersecurity: security monitoring and logging for production platforms, Kubernetes security (RBAC, network policies, admission control), vulnerability management, and handling incidents together with a security operations team.
Details
- Location: Hyderabad, Telangana, India (Hitec City Raidurg).
- Flexible work arrangement: Hybrid.
- No travel required.
- Domestic relocation applies; visa sponsorship not offered.
- Employee status: Regular.
- Job posting end date: 11/24/2026.
Read the full description and apply on the company’s own careers page.