Data Engineer

Overview

As a Data Engineer, you will build and maintain data systems including pipelines, data warehouses, and data lakes to keep data accurate, accessible, and secure.

What you'll do

• Build and maintain data architecture and pipelines for durable, complete, consistent data processing.
• Design and implement data warehouses and data lakes for data volume and velocity while meeting security requirements.
• Develop processing and analysis algorithms suitable for data complexity and scale.
• Collaborate with data scientists to build and deploy machine learning models.
• Lead and supervise a team, allocating work and coordinating resources (if leadership responsibilities apply).
• Ensure delivery of operational results, escalating policy/procedure breaches appropriately.
• Take ownership for embedding new risk-mitigation policies/procedures and strengthening controls.

What you'll need

• In-depth technical knowledge and experience in your area of expertise.
• Thorough understanding of underlying principles and concepts in the assigned expertise area.
• Advanced SQL skills, including query writing and performance tuning.
• Enterprise data warehousing experience with Teradata.
• Proficiency in distributed data processing with PySpark.
• Familiarity with big data tools and architecture using Hadoop.
• Strong Linux shell scripting skills for automation and data handling.

Nice to have

• AWS awareness and understanding of cloud-based data platforms and services.
• Prior exposure to AWS, especially for data migration or analytics on AWS.

Details

• Role location: Pune.