PwC logo

Senior Associate – Databricks Senior Data Engineer, Advisory

PwC
NewPosted today

LOCATION

Bengaluru Millenia · Onsite

EXPERIENCE

5 - 7 Years

TYPE

FullTime

Job description

Overview

Hands-on Databricks Data Engineer role at PwC's Advisory line, designing, building and optimizing scalable data pipelines on the Databricks Lakehouse.

What you'll do

  • Build reliable, scalable batch and streaming pipelines using PySpark and Databricks Workflows/Jobs.
  • Implement Delta Lake best practices: schema enforcement/evolution, ACID transactions, time travel, OPTIMIZE/ZORDER, and VACUUM.
  • Develop Delta Live Tables (DLT) or Structured Streaming pipelines with Auto Loader for ingestion.
  • Write robust modular code in Python (packaging, logging, configuration, error handling).
  • Write efficient and scalable SQL queries for data extraction and reporting.
  • Optimize Spark jobs (partitions, joins, bucketing, caching, AQE, broadcast hints, file sizing).
  • Work with SQL Warehouses, Photon, DBR versions, cluster policies, and pools.
  • Orchestrate Workflows/Jobs: tasks, dependencies, parameters, retries, alerts, schedules.
  • Apply Unity Catalog: permissions, data lineage, audit, external locations, shares; catalog-first design.
  • Ensure idempotent, repayable, and incremental pipelines with proper checkpoints and watermarks.
  • Implement data quality checks (expectations tests in DLT/Great Expectations) and unit/integration tests for pipelines.
  • Ensure data quality, integrity, and governance across the pipeline.
  • Work with CI/CD tools to automate deployment and testing of data workflows.
  • Work with stakeholders to define data SLAs, schemas, and interface contracts.
  • Document pipelines, data sets (data dictionary, lineage), and usage guidelines.

What you'll need

  • 5 to 7 years of experience.
  • Bachelor of Technology (BE/BTech).
  • Databricks certified Data Engineer Associate.
  • Strong SQL, Python and PySpark skills.
  • Understanding of Delta Lake.
  • Experience with production-grade ETL/ELT and orchestration.
  • Hands-on knowledge of Azure ADF and ADLS.
  • Primary skills: Databricks Platform, PySpark, SQL, Data Modelling, Python (scripting, data manipulation), Azure (ADF, ADLS).

Nice to have

  • Analytical mindset with strong problem-solving ability.
  • Secondary skills: CI/CD (Git, Jenkins, Azure DevOps, etc.).

Details

  • Location: Bengaluru Millenia.
  • Management level: Senior Associate.
  • Visa sponsorship: No.
  • Government clearance required: No.
  • Job posting end date: September 17, 2026.

Read the full description and apply on the company’s own careers page.

Stay safe

Hiring on Abekus is free for applicants

We never charge a fee, and employers are prohibited from doing so. If a recruiter asks for payment, please report them right away.

Senior Associate – Databricks Senior Data Engineer, Advisory