Senior Data Engineer, Data Engineering

Overview

Senior Data Engineer on the AI Network Services Team (AI & Decision Product Enablement Program), focused on data unification and curation across data assets to support data-driven cyber products, services and insights.

What you'll do

• Perform data ingestion, aggregation and processing to drive and enable relevant insights from available data sets.
• Leverage platforms such as Cloudera Data Engineering (CDE), Cloudera AI (CAI), Databricks and AWS to deliver product engineering solutions.
• Partner with Product Management, Data Science, Platform Strategy and Technology teams on data needs and requirements to deliver data solutions that generate business value.
• Manipulate and analyze complex, high-volume, high-dimensionality data from varying sources using a variety of tools and data analysis techniques.
• Identify innovative ideas and deliver proof of concepts and prototypes to meet existing and future needs, and propose new products, services and enhancements.
• Integrate new data assets that increase the value proposition for customers and enhance existing solutions and services.
• Analyse large volumes of transaction and product data to generate insights and actionable recommendations to drive business growth.
• Collect and synthesize feedback from clients, development, product and sales teams for new solutions or product enhancements.
• Apply knowledge of metrics, measurements and benchmarking to complex and demanding solutions.

What you'll need

• Good understanding of Python, including Pandas, NumPy, PySpark and Impala.
• Experience in data analysis, development and testing on Hadoop and Databricks cloud platforms.
• Strong SQL and higher-level programming languages, with solid knowledge of data mining, machine learning algorithms and tools.
• Exposure to collecting and/or working with data, including standardizing, summarizing, offering initial observations and highlighting inconsistencies.
• Experience in hands-on data modeling, programming, querying, data mining and report development using large volumes of granular data to deliver business intelligence and custom reporting solutions.
• Strong understanding of the application of analytical methods and data visualization to support business decisions.
• Ability to understand complex operational systems and analytics/business intelligence tools for delivering information products and analytical offerings to a large, global user base.
• Ability to work in a fast-paced, deadline-driven environment as part of a team and as an individual contributor.
• Ability to move between business, analytical and technical teams and articulate solution requirements for each group.

Nice to have

• Experience with data integration tools (ETL/ELT), such as Apache NiFi, Azure Data Factory, Pentaho or Talend.
• Experience with graph databases.

Details

• Location: Pune, India.
• Corporate security responsibility: abide by Mastercard's security policies and practices, ensure confidentiality and integrity of accessed information, report suspected information security violations or breaches, and complete periodic mandatory security trainings.