Big Data Engineer - Python and Spark

Overview

Trainee Data/Information Mgt Analyst responsible for applying technical experience to analyze information, solve problems, and deliver quality tasks within the team’s objectives.

What you'll do

• Apply acquired technical experience to carry out assigned tasks.
• Make evaluative judgments based on analysis of factual information.
• Resolve problems by selecting solutions using precedents.
• Redesign costly compute and data processes to best development standards.
• Work with large datasets and data warehouses to pull data using programs and code.
• Use data preprocessing and application engineering skills.
• Present complex business problems to management in a structured, simpler form.

What you'll need

• Master’s or Engineering degree.
• 0–2 years of experience in Big Data systems, Hive, Hadoop, Spark (Python/Scala) and cloud-based data management.
• Hands-on experience with Unix scripting, Python and Scala programming.
• Strong experience in SQL.
• Experience with working on unstructured, undocumented code and turning it into redesigned best-in-class code.
• At least 3 years of experience designing software systems with intense computational needs across real time and batch processing.
• Experience and understanding of supervised and unsupervised machine learning techniques.

Nice to have

• Experience with data ingestion and ETL tools such as Talend.
• Experience with performance management tooling such as Pepper data.
• Experience with the Cloudera stack.
• Previous related experience preferred.

Details

• Work model: experience working in onsite and offsite delivery model.