Unified Data Platform Architect

Overview

Principal Software Engineer responsible for architecting and evolving large-scale data platforms, distributed systems, backend services, and cloud-native infrastructure. The role provides technical leadership across systems and teams while remaining hands-on.

What you'll do

• Define architecture and long-term direction for large-scale data platforms.
• Design scalable backend services and distributed data infrastructure.
• Build batch and real-time processing architectures with Spark and Flink.
• Establish ingestion, transformation, orchestration, storage, and serving patterns.
• Design workflow orchestration capabilities using Airflow.
• Lead reliability, performance, observability, and infrastructure-efficiency improvements.
• Mentor engineers and guide architecture reviews across teams.

What you'll need

• 10+ years of software engineering experience.
• Deep expertise in Java and strong software engineering fundamentals.
• Proficiency in Python for data engineering or platform development.
• Experience with Spark, Hadoop, HDFS, Hive, and YARN.
• Experience with Flink, Airflow, Kubernetes, containers, and cloud-native architectures.
• Deep understanding of distributed-systems concepts and batch-versus-streaming tradeoffs.
• Experience driving technical decisions across multiple teams or major platform initiatives.

Nice to have

• Experience with petabyte-scale datasets or billions of events per day.
• Experience with Apache Iceberg and lakehouse technologies.
• Experience designing multi-tenant data platforms.
• Knowledge of data governance, lineage, quality, and observability.

Details

• Location: Bengaluru, India.