R

Senior Data Engineer

Radcube
NewPosted today

LOCATION

Shaikpet · Onsite

EXPERIENCE

6 - 10 Years

TYPE

FullTime

SKILLS REQUIRED

Data EngineeringETL PipelinesData GovernanceData QualityData ReconciliationCI/CDGitSQLPySpark

Job description

Overview

Build and operationalize data engineering pipelines using AWS Glue, PySpark and Delta Lake to create governed, high-quality enterprise datasets.

What you'll do

  • Develop AWS Glue / PySpark transformations from Raw to Refined datasets in Amazon S3.
  • Implement schema enforcement, type conversion, PII tokenization, deduplication and merges.
  • Develop Delta Lake pipelines and, when required, Gold datasets and Redshift loads.
  • Orchestrate jobs using Airflow or AWS Step Functions with completeness and failure checks.
  • Register datasets in Unity Catalog and support governance, metadata and consumer access.
  • Perform reconciliation, performance tuning, unit testing and production troubleshooting.
  • Prepare CI/CD artifacts, deployment configuration, runbooks and technical documentation.

What you'll need

  • Python, PySpark, Spark SQL and strong SQL.
  • AWS Glue, Amazon S3, Delta Lake and Databricks.
  • ETL/ELT, data modelling and data-quality validation.
  • Airflow or AWS Step Functions and failure recovery.
  • Git, CI/CD, data reconciliation and production support.

Nice to have

  • Unity Catalog, Amazon Redshift, Immuta, ServiceNow integration and regulated-data experience.

Details

  • Location: Hyderabad, India.

Read the full description and apply on the company’s own careers page.

Stay safe

Hiring on Abekus is free for applicants

We never charge a fee, and employers are prohibited from doing so. If a recruiter asks for payment, please report them right away.

Senior Data Engineer