Data Engineer- Azure Databricks- Senior Associate

Overview

Support data platform modernization for PwC's Advisory practice, focused on Azure Databricks, cloud data pipelines, ETL development, and analytics enablement, while mentoring junior team members.

What you'll do

• Build Azure Databricks data pipelines and scalable data engineering solutions across cloud environments
• Design and maintain data integration, transformation, and validation workflows for modern analytics platforms
• Optimize data models, query performance, and warehouse structures to support faster reporting and analysis
• Apply Apache Airflow, Azure Data Factory, and Databricks tools to orchestrate dependable batch and streaming processes
• Collaborate with client stakeholders to translate business needs into technical data architecture and delivery plans
• Support data modernization efforts that migrate legacy platforms to cloud-native architectures
• Review code, pipeline logic, and data quality controls to validate consistency and support issue resolution
• Mentor junior team members on engineering methods, platform usage, and delivery practices

What you'll need

• At least a Bachelor's degree
• At least 4 years of experience
• Oral and written proficiency in English
• 4+ years of experience working in Azure ADLS, Data Bricks, Data Flows, HDInsight, Azure Analysis services
• Awareness of data management concepts and data modelling
• Good knowledge of Big Data frameworks and related technologies
• Experience in Hadoop and Spark (mandatory)
• Experience extracting data from different source systems like flat files, XML sources, JSON, Parquet files, Big data appliances, RDBMS, etc.
• Clear understanding of the DW lifecycle and contribution to preparing Detail Level Design documents, Unit Test plans, and code review reports
• Strong SQL, including joins, groups, functions and stored procedures
• Experience in unit and integration test creation and verification
• Proficiency in creating documentation for project architecture, end-user manuals, and operations hand-off guides
• Good understanding of data migration processes, methods and project lifecycle
• Good Unix shell scripting skills
• Strong analytical and troubleshooting skills
• Well versed with quality processes and implementation
• Excellent communication skills
• Strong understanding of cloud data migration processes, methods and project lifecycle
• Good knowledge of application DevOps tools (Git, CI/CD frameworks)
• Experience in Jenkins or GitLab with rich experience in source code management (Code Pipeline, Code Build and Code Commit)

Nice to have

• Certifications in Azure / Databricks
• Experience building stream-processing systems using solutions such as Storm or Spark Streaming
• Experience in Big Data ML toolkits such as Mahout, SparkML, or H2O
• Experience in Azure Functions
• Knowledge of Python
• Preference for at least one of these fields of study: Management Information Systems, Computer and Information Science, Systems Engineering, Electrical Engineering, Chemical Engineering, Industrial Engineering, Mathematics, Statistics, Mathematical Statistics
• Demonstrated Azure Databricks delivery across data modernization engagements
• Application of Databricks, Azure Data Factory, and Airflow in pipeline builds
• Translating complex data requirements into scalable engineering solutions
• Analyzing data architecture, modeling, and optimization opportunities
• Mentoring junior team members while managing assigned workstreams

Details

• Location: Bangalore, India (Acceleration Center India)
• Level: Senior Associate