Data Engineer

Overview

As a Data Engineer, design, develop and maintain data solutions for data generation, collection and processing, including data pipelines, data quality processes and ETL processes across systems. Develop scalable, high-performing custom software solutions and collaborate with multiple teams using modern frameworks and agile practices.

What you'll do

• Design, develop and maintain data solutions for data generation, collection and processing.
• Create data pipelines and ensure data quality.
• Implement ETL processes to migrate and deploy data across systems.
• Use Python/PySpark and SQL in data engineering work.
• Use AWS services including EMR, EC2, S3 and Glue for ETL processes.
• Perform as a subject matter expert.
• Collaborate with and manage the team to perform.
• Take responsibility for team decisions.
• Engage with multiple teams and contribute to key decisions.
• Provide solutions to problems for the immediate team and across multiple teams.
• Facilitate knowledge-sharing sessions to enhance team capabilities.
• Mentor junior team members to support their professional growth.
• Code and enhance software components.
• Collaborate with various teams to deliver scalable and high-performing solutions.
• Stay updated with industry trends and best practices to improve the software development process.

What you'll need

• Minimum 5 year(s) of experience.
• Minimum 6 years of experience in AWS BigData.
• Strong experience in Data Engineering, ETL process and Data quality process.
• Hands-on experience in Python/PySpark and SQL.
• Hands-on experience in AWS services including EMR, EC2, S3 and Glue for ETL process.
• Proficiency in AWS BigData.
• Experience with cloud computing platforms and services.
• Strong understanding of data processing frameworks such as Apache Hadoop and Apache Spark.
• Familiarity with data storage solutions including Amazon S3 and Amazon Redshift.
• Ability to implement data pipelines and ETL processes.
• Good communication skills.
• 15 years full-time education.

Details

• Location: Mumbai.
• The position is based at the Mumbai office.
• Shift time is 1 PM to 10:30 PM.