Data Engineer

Overview

The Applications Development Intermediate Programmer Analyst contributes to applications systems analysis and programming activities and participates in establishing and implementing new or revised application systems and programs in coordination with the Technology team.

What you'll do

• Translate application storyboards and use cases into functional applications.
• Design, build, and maintain efficient, reusable, and reliable Python/Scala code.
• Ensure the best possible performance, quality, and responsiveness of applications.
• Identify bottlenecks and bugs, and devise solutions to these problems.
• Develop high performance and low latency components to run Spark clusters.
• Interpret functional requirements into design approaches that can be served through a Big Data platform.
• Collaborate and partner with Global Teams based across different locations.
• Propose best practices and standards.
• Perform testing of software prototypes and transfer them to the operational team.
• Process data using Hive, Impala and HBASE.
• Perform analysis of large data sets and derive insights.

What you'll need

• Strong experience with one or more programming languages, including Python or Spark, with 4+ years of experience.
• Demonstrated analytical and problem-solving skills.
• Application performance tuning and troubleshooting experience in the Big Data domain.
• Comfort working with large data volumes and a firm understanding of logical data structures and analysis techniques.
• Experience with Big Data technologies including HDFS, MapReduce, YARN, Apache Spark and Hive.
• Familiarity with data formats including Avro, Parquet, CSV and JSON.
• Strong systems analysis, design and architecture fundamentals.
• Experience with unit testing and other SDLC activities.
• Understanding of fundamental design principles behind a scalable application.
• Experience working with UNIX/Linux and shell scripting.
• Good knowledge of database principles and practices with SQL.
• Application performance tuning and troubleshooting experience in the Big Data domain.

Nice to have

• Agile/Scrum methodology experience.
• Experience with SCMs such as Git and tools such as Jira.
• Experience with NoSQL databases.
• Familiarity with build tools such as Maven and continuous integration tools such as Jenkins or TeamCity.
• Familiarity with cloud and container technologies.
• Experience with real-time data processing using Kafka.

Details

• Location: Pune, Maharashtra, India.
• Employment type: Full time.