Senior Associate – Databricks Engineer, Advisory

Overview

Design and develop robust data solutions, infrastructure and systems that enable efficient data processing and analysis for clients. Build data pipelines, data integration and data transformation solutions using Databricks and related technologies.

What you'll do

• Design and develop robust data solutions for clients.
• Design and build data infrastructure and systems for efficient data processing and analysis.
• Develop and implement data pipelines, data integration and data transformation solutions.

What you'll need

• 4-8 years of experience required.
• B.Tech / M.Tech / MBA / MCA.
• Strong hands-on experience in Databricks.
• Strong coding skills in Python, PySpark, and SQL.
• Experience building ETL/ELT pipelines.
• Hands-on experience with Delta Lake.
• Experience with Unity Catalog, Delta Live Tables (DLT), Auto Loader, Databricks Workflows / Lakeflow.
• Knowledge of Structured Streaming.
• Good understanding of Lakehouse architecture.
• Experience on AWS / Azure.
• Strong experience in Spark performance tuning / optimization.
• Good communication skills.
• PySpark.

Nice to have

• Experience with Photon.
• Exposure to Databricks SQL.
• Exposure to MLflow / Databricks ML.
• Experience with CI/CD, Git.
• Knowledge of Airflow / ADF / other orchestration tools.
• Databricks certification is a plus.
• Domain experience in healthcare is preferred.

Details

• Location: Bengaluru Millenia.
• Available for work visa sponsorship: No.
• Government clearance required: No.