Python and Apache Spark Big Data Engineer - Assistant Vice President

Overview

Assistant Vice President role in Applications Development at Citi, building Python and Apache Spark Big Data applications.

What you'll do

• Translate application storyboards and use cases into functional applications.
• Design, build, and maintain efficient, reusable, and reliable Python/Scala code.
• Ensure the best possible performance, quality, and responsiveness of the applications.
• Identify bottlenecks and bugs, and devise solutions to these problems.
• Develop high performance & low latency components to run Spark clusters.
• Interpret functional requirements into design approaches that can be served through the Big Data platform.
• Collaborate and partner with Global Teams based across different locations.
• Propose best practices and standards.
• Perform the testing of software prototypes and transfer to the operational team.
• Process data using Hive, Impala & HBASE.
• Perform analysis of large data sets and derive insights.

What you'll need

• 8 to 12 years of experience.
• Strong experience with one or more programming languages (Python/Spark).
• Demonstrated analytical and problem-solving skills.
• Application performance tuning and troubleshooting experience, and implementation of these skills in the Big Data domain.
• Comfortable working with large data volumes, with a firm understanding of logical data structures and analysis techniques.
• Experience in Big Data technologies like HDFS, MapReduce, YARN, Apache Spark, Hive.
• Familiarity with data formats like Avro, Parquet, CSV, JSON.
• Strong systems analysis, design and architecture fundamentals, unit testing and other SDLC activities.
• Understanding of fundamental design principles behind a scalable application.
• Experience working with UNIX/LINUX and shell scripting.
• Good knowledge of database principles and practices with SQL.

Nice to have

• Agile/Scrum methodology experience.
• Experience in SCMs like GIT and tools like JIRA.
• Experience in NoSQL databases.
• Familiarity with build tools such as Maven and continuous integration such as Jenkins/Team City.
• Familiarity with cloud and container technologies.
• Experience with real-time data processing using Kafka.

Details

• Full time.
• Location: Pune, Maharashtra, India.