Overview
Principal Data Engineer (Lead Data Engineer) providing technical leadership for scalable data solutions in a Lakehouse architecture.
What you'll do
- Lead design, development, and implementation of scalable Lakehouse data engineering solutions.
- Define technical approaches for ingestion, transformation, integration, and consumption.
- Lead complex ETL/ELT initiatives across multiple source systems and business domains.
- Design and oversee pipelines using Azure Databricks, PySpark, SQL, and Delta Lake.
- Drive migration of existing data assets and transformation logic into the Lakehouse.
- Establish data engineering standards and reusable frameworks, including layered architecture (Bronze/Silver/Gold).
- Implement data quality, validation, reconciliation, monitoring, and exception-handling processes.
What you'll need
- 10-15 years professional experience.
- Strong expertise in Lakehouse architecture and modern data platform concepts.
- Extensive experience with Azure Databricks.
- Strong expertise in Apache Spark, PySpark, and Delta Lake.
- Expert-level proficiency in SQL for complex transformations and performance optimization.
- Strong hands-on experience with Python and PySpark.
- Knowledge of data governance (metadata, lineage, data quality) and security/access management.
Details
- Location: Chennai, India.
Read the full description and apply on the company’s own careers page.
NewPosted yesterday
