Azure Databricks Data Engineer – Apache Spark & Azure Data Factory

Overview

Build and optimize cloud-based data platforms using Azure and Databricks. Deliver reliable, secure, and cost-efficient data pipelines supporting enterprise business objectives.

What you'll do

• Design, develop, and maintain ETL/ELT pipelines using Spark and Databricks.
• Build batch and streaming data solutions.
• Develop Azure Data Factory pipelines and automate deployments.
• Optimize Databricks clusters, caching, queries, and cloud resource usage.
• Secure data and credentials using Key Vault and service principals.
• Monitor pipelines, resolve issues, and improve reliability.
• Collaborate with engineering teams, stakeholders, and platform teams.
• Document solutions and support continuous technical improvement.
• Develop PySpark, Python, SQL, and shell-based solutions.
• Build and monitor ADF pipelines and Databricks Workflows.
• Tune Spark jobs and manage cluster configurations.
• Investigate failures and resolve production issues.
• Participate in Agile meetings, design reviews, and code reviews.
• Produce pipeline documentation, test evidence, and support materials.
• Make technical decisions within approved architecture and security standards.

What you'll need

• 10+ years in data engineering or Azure cloud technologies.
• 3+ years optimizing Databricks workloads.
• Experience with cluster sizing, caching, query optimization, Spark, ADF, Python, SQL, and Linux.
• Bachelor’s or master’s degree in Computer Science, Information Technology, Mathematics, Data Engineering, or a related field; equivalent professional experience may be considered.
• Azure, Azure Databricks, Azure Data Factory, and Azure Data Lake.
• Azure Key Vault, service principals, and Azure CLI.
• Apache Spark, PySpark, Python, T-SQL, and Linux/Shell scripting.
• Databricks Jobs, Workflows, Azure Pipelines, and CI/CD.
• Git and cloud-based data engineering tools.
• Data lakes, ETL/ELT, SQL, data modeling, partitioning, and optimization.
• Apache Spark, Spark SQL, PySpark, Databricks Jobs, and Workflows.
• Git, CI/CD, Azure Pipelines, testing, monitoring, and Agile practices.
• Secure credential management, role-based access, and least-privilege principles.
• Completion of Synechron security, compliance, and project training is required.
• Ongoing learning in Azure, Databricks, Spark, security, and cloud efficiency is expected.
• Analytical problem-solving and root-cause analysis.
• Effective teamwork and technical leadership.
• Clear communication with technical and business stakeholders.
• Ability to manage priorities, deadlines, and dependencies.
• Adaptability and willingness to learn new technologies.
• Focus on secure, maintainable, reliable, and cost-efficient solutions.

Nice to have

• Agile delivery experience.
• PL/SQL.
• Data-quality, monitoring, and infrastructure automation tools.
• Data governance and lineage.
• Infrastructure as code and cloud cost management.
• Delta-based storage and streaming frameworks.
• Enterprise data platform or regulated-industry experience.
• Azure or Databricks certification.
• Equivalent practical experience, certifications, or substantial project work may be considered.

Details

• Location: Pune - Kharadi (EON-II).