Lead Data Engineer

Overview

Lead the design, development, and scaling of secure data platforms, pipelines, and analytical ecosystems for enterprise data warehouse and lakehouse environments.

What you'll do

• Design and deploy scalable data pipelines using Apache Spark, Java or Scala, Hadoop, and cloud object storage.
• Build batch and near-real-time processing frameworks for petabyte-scale workloads.
• Implement data lineage, metadata, cataloguing, quality controls, and observability.
• Migrate legacy ETL and data warehouse workloads to cloud-native architectures.
• Lead requirements analysis, solution design, coding, testing, deployment, and production support.
• Mentor junior engineers and collaborate with technical and business stakeholders.
• Troubleshoot production incidents and implement long-term remediation.

What you'll need

• 10–12 years of experience delivering enterprise-scale data warehouse, data lake, or data lakehouse solutions.
• Experience delivering end-to-end data engineering projects in distributed computing environments.
• Experience migrating ETL and analytics workloads from on-premises to cloud-native environments.
• Strong experience with Apache Spark, Scala or Java, Hadoop, and object storage platforms.
• Experience with Apache Airflow, Apache NiFi, or similar scheduling frameworks.
• Strong SQL skills and experience with relational and NoSQL databases.
• Ability to work in Agile, distributed, matrix-based teams with strong ownership and communication.

Details

• Location: Pune, India