Manager – Databricks Engineer, Advisory

Overview

As a Manager – Databricks Engineer in Advisory, you will design and build data infrastructure and systems, develop data pipelines, and deliver data integration and transformation solutions for clients.

What you'll do

• Design and build data infrastructure and systems for efficient data processing and analysis.
• Develop and implement data pipelines.
• Develop and implement data integration solutions.
• Develop and implement data transformation solutions.

What you'll need

• 7-12 years of experience.
• B.Tech, M.Tech, MBA, or MCA.
• Strong hands-on experience in Databricks.
• Strong coding skills in Python, PySpark, and SQL.
• Experience building ETL/ELT pipelines.
• Hands-on experience with Delta Lake.
• Experience with Unity Catalog, Delta Live Tables (DLT), Auto Loader, and Databricks Workflows / Lakeflow.
• Knowledge of Structured Streaming.
• Good understanding of Lakehouse architecture.
• Experience on AWS / Azure.
• Strong experience in Spark performance tuning / optimization.
• Good communication skills.
• PySpark.

Nice to have

• Experience with Photon.
• Exposure to Databricks SQL.
• Exposure to MLflow / Databricks ML.
• Experience with CI/CD and Git.
• Knowledge of Airflow / ADF / other orchestration tools.
• Databricks certification is a plus.
• Domain experience in healthcare is preferred.

Details

• Location: Bengaluru Millenia.
• Government clearance is not required.
• Available for work visa sponsorship: No.