BigData-Hadoop/Scala Engineer, AVP

Overview

Design and develop scalable engineering solutions and high-performance, fault-tolerant data pipelines using Hadoop, Scala, Spark, and Google Cloud Platform technologies.

What you'll do

• Design, develop, and maintain high-performance data pipelines.
• Implement ETL data processing solutions from multiple sources.
• Collaborate with data scientists, analysts, and engineers on data requirements.
• Optimize data processing jobs for performance and cost efficiency.
• Ensure data quality, integrity, and security across data assets.
• Participate in code reviews and technical discussions.
• Mentor junior team members and stay current with big data practices.

What you'll need

• Significant experience across multiple engineering implementations.
• Strong proficiency in Java and/or Scala.
• Extensive experience with Apache Spark.
• Hands-on experience with GCP, including Airflow, Dataproc, and BigQuery.
• Understanding of data warehousing, ETL/ELT, and data modeling.
• Experience with version control systems such as Git.
• Strong problem-solving, communication, and collaboration skills.

Nice to have

• Proficiency in Python.
• Experience with Oracle or other relational databases.
• Unix/Linux scripting experience.
• Familiarity with AWS or Azure.
• Knowledge of CI/CD, DevOps, data governance, and data security.
• Financial services industry experience.
• Experience developing UI with Angular or React.

Details

• Location: Pune, India.
• Level: AVP.