Sr Associate – Databricks Data Engineer, Advisory

Overview

Design, build, and operate scalable data pipelines on the Databricks Lakehouse Platform, supporting analytics, reporting, and AI/ML workloads.

What you'll do

• Design and maintain end-to-end data pipelines using Databricks, PySpark, and Spark SQL.
• Implement batch and incremental processing with Delta Lake and multi-hop architecture.
• Develop and optimize Databricks notebooks, jobs, and workflows.
• Ingest, transform, and curate structured and semi-structured datasets.
• Ensure data quality, lineage, reliability, and governance.
• Optimize Spark jobs for performance and cost efficiency.
• Mentor junior engineers and provide production support.

What you'll need

• 7+ years of experience as a Data Engineer with strong Databricks expertise.
• Hands-on experience with Apache Spark, PySpark, and Spark SQL.
• Strong knowledge of Delta Lake and Lakehouse architecture.
• Advanced SQL skills.
• Experience with ETL/ELT patterns and data warehousing concepts.
• Experience with at least one cloud platform: Azure, AWS, or GCP.
• Bachelor's or Master's degree in Computer Science, Engineering, or a related field, with 60% or above.

Nice to have

• Experience with Unity Catalog and data governance.
• Exposure to Auto Loader or streaming frameworks.
• Experience with CI/CD for data pipelines.
• Python for data engineering and automation.
• Databricks Associate or Professional certification.

Details

• Location: Bengaluru.
• Management level: Senior Associate.