Data Engineer II

Overview

Data Engineer II role supporting multiple Data Analysts and Data Scientists day-to-day and assisting various data-driven requests across the company. Involves a wide variety of technical skills, from coding to infrastructure, from data analysis to database management.

What you'll do

• Build and maintain database technologies, environments, and applications, seeking opportunities for improvements and efficiencies
• Work with cloud data platforms such as Databricks and/or Cloudera, Iceberg/Delta tables
• Write SQL statement procedures to optimize SQL execution and query development
• Utilize technologies like Spark (Scala, Python), Apache Kafka, Trino
• Manage the deployment of data solutions that are optimally standardized and database updates to meet project deliverables
• Lead database security posture, proactively identifying security risks and implementing risk mitigation plans and control functions
• Support process improvement efforts to identify and test opportunities for automation and/or reduction in time to deployment
• Participate in the entire solution development cycle from requirement gathering to solution deployment
• Work with multiple stakeholders across teams on ad-hoc investigations, including large-scale data extraction, transformation, and analyses
• Support production code to produce comprehensive and accurate data artifacts
• Promote strategies to improve data modeling, quality, and architecture

What you'll need

• Experience working with Databricks and/or Cloudera platforms
• Skilled in Spark framework (Scala/Python), Apache Kafka, Spark Streaming
• Highly skilled in SQL and relational databases
• Experience developing and maintaining data pipelines for the extraction, transformation, and/or loading of large-scale data
• Experience working with large data sets, including implementing parallel processing and dealing with the limitations and challenges of terabytes of data
• Strong analytics skills for working with unstructured datasets
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement
• Experience with database administration and query optimization across different databases
• Experience with at least one programming language such as Python
• Capable of effectively communicating requirements and solutions to audiences with different skill sets
• Familiarity with the core requirements of data analysis and/or presentation
• Focused, innovative, and self-sufficient problem solver who can independently and creatively deep-dive into an issue and solve it

Nice to have

• Experience with AWS: S3, EMR, Ozone, Iceberg
• Experience with Python, Javascript, PLSQL, regex
• Experience with Jenkins, Datadog, Splunk, Dynatrace

Details

• Location: Pune, India