Software Engineer - Model API Platform

Overview

Own major parts of Sarvam's model API platform (ASR, TTS, Vision, LLM, Translate), end to end, working closely with the Staff Engineer who leads its architecture. The platform handles tens of millions of API calls daily over HTTP and WebSockets, including metering, billing, prepaid wallets and rate-limiting.

What you'll do

• Own one or more major platform areas, such as the WebSocket streaming layer, a speech recognition pipeline, or a text to speech pipeline: build, operate and improve them.
• Turn loosely defined goals into RFCs, milestones and shipped results, and write down decisions and trade-offs.
• Be accountable after launch: on-call, incident response, root-cause analysis and follow-through on fixes.
• Make the call when a decision is needed, and own the outcome.
• Dissect existing services to understand the system, and tune the right knobs to optimize latency and performance.
• Bring services across without regressions in customer-facing APIs, latency or reliability.
• Build and operate streaming infrastructure: WebSocket connections, long-lived sessions, queue-based batch processing, backpressure and low-latency model invocation.
• Engineer audio pipelines: TTS chunking around model context limits, sample-rate conversion and format encoding, and VAD-based silence detection for ASR.
• Hold services to sub-100ms latency SLOs in a multi-tenant environment.
• Reason from constraints (latency budgets per hop, throughput, cost per request, failure modes, tenant isolation) and use back-of-the-envelope estimates to guide decisions.
• Build in observability from day one: logging, metrics and tracing.
• Strengthen the integration testing harness so the team can ship fast without breaking customer-facing APIs.
• Partner with the Inference and MLOps teams on making model serving fast, reliable and operable.
• Mentor junior engineers through RFC reviews, code review and pairing.
• Practice design-first, documentation-first engineering: RFCs before code.

What you'll need

• 4–6 years of building and operating production backend systems.
• At least one service you owned end to end under real load, including what broke and what you changed.
• Strong distributed-systems fundamentals: concurrency, networking, queues, caching, consistency and failure modes.
• Experience with low-latency or high-throughput services, and comfort reasoning about p95 and p99 latency.
• Solid fundamentals with a relational database and a cache, such as PostgreSQL and Redis: schema design, query performance and caching strategy.
• Fluency in at least one backend language and willingness to work in both Python and Golang.
• Experience running services with containers, CI/CD and production monitoring.
• Daily, hands-on use of AI tools in your engineering workflow, with concrete examples of what you automated or sped up.
• A bias for action: fast on reversible decisions, careful on irreversible ones.
• Clear writing, especially RFCs and design docs that others can act on.

Nice to have

• Voice AI or real-time audio experience: WebSockets, WebRTC, streaming ASR or TTS, or audio processing with FFmpeg or pyav, codecs, sample rates and VAD.
• Production experience in Go, or in Python with FastAPI.
• Experience serving LLM, ASR, TTS or vision models in production.
• Experience building metering, billing, wallet or payments infrastructure.
• Running services on Kubernetes in production.
• Time at an early- or growth-stage startup.

Details

• Location: Bengaluru.