Data Engineer (Python/PySpark)

Overview

Build and support scalable data engineering solutions, including Data Marts, ETL pipelines, and large-scale data processing, using Python and PySpark.

What you'll do

• Design, develop, and maintain scalable ETL/Data Pipeline solutions using Python and PySpark.
• Build and optimize Data Marts and data transformation processes.
• Perform end-to-end SDLC activities including development, UAT support, bug fixes, production deployment, and post-production support.
• Analyze and debug data issues using Oracle SQL, PySpark, and data analysis techniques.
• Work with structured, semi-structured, and unstructured data.
• Collaborate with cross-functional teams to ensure data quality, testing, and CI/CD implementation.
• Support production environments and troubleshoot data pipeline issues.

What you'll need

• 5+ years of experience in building Data Marts, ETL pipelines, and large-scale data processing solutions.
• Strong hands-on experience in Python, PySpark, Spark, and Pandas.
• Expertise in ETL development and Data Warehousing concepts.
• Strong knowledge of Oracle SQL, data analysis, and query optimization.
• Experience with Hadoop, MapReduce, Hive, SQL, and NoSQL databases.
• Familiarity with Jupyter Notebook, CI/CD pipelines, testing, and data validation.
• Good understanding of software engineering best practices and coding standards.

Nice to have

• Banking domain experience is highly preferred.
• Preferred domain: Banking / Financial Services.

Details

• Location: Bengaluru - Bellandur (GTP), Bangalore (BLR GTP).