Senior Software Engineer - Data Infrastructure

Overview

Own Sarvam's central data platform end to end, covering ingestion, streaming, the ClickHouse Cloud warehouse, transformations, access control and audit that every product depends on.

What you'll do

• Build and run the ingestion path: the service products send events to, the Kafka behind it, and the pipes that land data in ClickHouse within seconds.
• Own schema validation, rate limits, dead-letter handling and the client SDKs.
• Own the warehouse on ClickHouse Cloud: design tables for the read path (sorting keys, partitions, materialized views, TTL), separate compute per workload, set per-user limits and per-tenant isolation, and attribute cost to the team that spent it.
• Build reliable data pipelines: dbt models and Airflow DAGs with the BI engineers, CDC from product databases, and sync and backfill jobs that are safe to re-run.
• Solve data problems product teams bring, such as a late source, a number that doesn't match or a query that has slowed down.
• Make the platform self-serve: a registry of every dataset with its owner, schema, freshness and access, and provisioning and permissions as code.
• Run it in production: everything as Terraform and Helm with metrics, traces and alerts; be on call, handle incidents end to end, write the RCA and fix the root cause.
• Help decide what the platform should do next and build it, whether a new source or sink, or a new way for teams to use the data.

What you'll need

• 2–5 years of experience building, shipping and running large-scale, data-intensive systems in production, ideally at a startup where you owned things and were on call for them.
• Deep expertise in at least one of: an OLAP database (ClickHouse preferred; Druid, BigQuery or Redshift also fine) or Kafka / event streaming.
• Understanding of how a columnar store writes, merges and scans data, or how a partitioned log gives ordering and delivery guarantees.
• Experience with large-scale messaging systems like Kafka or RabbitMQ: partitions, consumer groups, delivery guarantees, backpressure.
• Good at Go, or good at Python.
• Knowledge of data-system problems: schema evolution, idempotency, backpressure, partitioning, retention and TTL, CDC.
• Built or run at least one of: an event pipeline, a transformation layer (dbt, Spark or SQL on Airflow), a provisioning or control-plane service, or a multi-tenant data service with access control.
• Comfortable with AWS, Kubernetes and Terraform in production.
• Test before shipping, debug with metrics and traces, write RCAs, and document what you build.

Nice to have

• ClickHouse in production: MergeTree, materialized views, projections, quotas, settings profiles, row policies, ClickHouse Cloud.
• Kafka in depth: MSK, Kafka Connect, schema registry (Glue or Confluent), Avro.
• dbt and Airflow at scale.
• Audit or security data systems: append-only logs, integrity checks, tenant isolation.
• OpenTelemetry, LLM observability (Langfuse or similar), eval pipelines.
• Built an internal platform that other teams adopted.

Details

• Location: Bengaluru.
• Level: Software Engineer / Senior Software Engineer.