Data Engineer

Overview

Design, develop and maintain data solutions for data generation, collection and processing, including data pipelines, data quality and ETL processes to migrate and deploy data across systems.

What you'll do

• Perform independently and become an SME.
• Actively participate in and contribute to team discussions.
• Contribute solutions to work-related problems.
• Collaborate with cross-functional teams to understand data requirements and deliver scalable solutions.
• Monitor and troubleshoot data pipelines to ensure reliability and performance.
• Document data processes and workflows to maintain clarity and support knowledge sharing.
• Assist junior team members by providing guidance and support in their tasks.

What you'll need

• Must have: proficiency in PySpark.
• Minimum 3 years of experience in PySpark.
• 15 years full time education.
• Experience building and optimizing data pipelines and workflows using PySpark.
• Strong knowledge of data processing frameworks and distributed computing concepts.
• Familiarity with data storage solutions and data modeling techniques.
• Ability to work with large datasets and ensure data quality and integrity.
• Experience in troubleshooting and debugging data processing issues.

Details

• Based at the Chennai office.