Staff Engineer - Big Data

Overview

Staff Engineer - Big Data to design, develop, and maintain scalable data pipelines for large-scale structured and unstructured datasets.

What you'll do

• Design, develop, and maintain scalable data pipelines using PySpark and Apache Spark.
• Process and analyze large-scale datasets in distributed environments.
• Build real-time analytics on cloud and edge devices.
• Solve data and architectural problems using advanced technologies.
• Collaborate cross-functionally with data scientists, data engineering, and firmware controls teams.
• Implement data wrangling, transformation, and processing solutions.

What you'll need

• 3+ years of relevant experience.
• Strong Java and Scala programming/debugging ability and understanding of design patterns.
• Hands-on understanding of Kafka, Spark, Flink, Hadoop, or HBase internals (experience in one or more).
• Experience in performance tuning and debugging Spark jobs.
• Good understanding of distributed computing principles.
• Experience with cloud computing platforms such as AWS, GCP, or Azure (beneficial).
• Knowledge of data lakes/warehousing concepts, SQL, and NoSQL databases, and REST APIs or gRPC (good to have).

Details

• Location: Bangalore and Mumbai, India.