LOCATION

Mumbai · Onsite

EXPERIENCE

Not specified

SALARY

Negotiable

SKILLS REQUIRED

distributed systemsObservabilityProduction SupportPythonPerformance EngineeringIncident response

Job description

Overview

AI Platform Engineer role focused on first- and second-line support for an LLM gateway platform and maintaining supporting infrastructure.

What you'll do

Provide first- and second-line support for the LLM gateway platform, troubleshooting issues from engineering and business users.
Monitor and maintain underlying infrastructure to ensure availability, stability, and predictable performance.
Troubleshoot model-serving backends and provider integrations, including latency, throughput, error-rate, and capacity issues.
Triage incidents affecting model availability and drive resolution, involving vendors when needed.
Support tooling on top of the LLM gateway, including integrations, developer workspaces, coding assistants, and API clients.
Coordinate with platform engineering, cloud infrastructure, and end-user teams to resolve incidents and minimize disruption.

What you'll need

Experience in production support, SRE, or platform operations with strong end-to-end issue ownership.
Strong Linux and Windows system administration skills.
Proficiency scripting/automation in Python, Bash and/or PowerShell.
Solid experience with relational databases such as PostgreSQL or SQL Server.
Practical understanding of monitoring and observability (metrics, logs, traces, dashboards, alerting).
Comfort debugging distributed API-driven services (HTTP semantics, timeouts, retries, connection resets, caching, latency percentiles).
Familiarity with large language model hosting concepts (inference APIs, gateways/proxies, prompt/context handling, token accounting, streaming, prompt caching).

Details