Specialist - Data Platform Engineering

Overview

Specialist - Data Platform Engineering to build and operate scalable, reliable, high-performance data platforms.

What you'll do

• Develop, maintain, and optimize batch and streaming data pipelines using Apache Spark and Apache Flink.
• Build scalable ETL/ELT workflows for processing large datasets.
• Manage Databricks workspaces, clusters, jobs, notebooks, and workflows.
• Monitor, troubleshoot, and optimize Databricks jobs for performance and cost efficiency.
• Work with Apache Kafka for event-driven processing and streaming applications.
• Configure, deploy, and troubleshoot Kafka Connect connectors for data source/sink integration.
• Provide production support, incident response, and root cause analysis for data platforms.

What you'll need

• 3–4 years of experience in software engineering or data engineering.
• Strong programming skills in Java, Scala, or Python.
• Hands-on experience with Apache Spark for large-scale data processing.
• Knowledge of Apache Flink for stream processing.
• Experience with Databricks, including cluster management and job orchestration.
• Good understanding of Apache Kafka concepts including topics, partitions, producers, and consumers.
• Experience with Git and CI/CD practices.

Nice to have

• Experience with Delta Lake and Lakehouse architecture.
• Knowledge of cloud platforms such as AWS, Azure, or GCP.
• Familiarity with orchestration tools like Airflow or Argo Workflows.
• Experience with monitoring tools such as Prometheus and Grafana.
• Understanding of Kubernetes and containerized deployments.
• Knowledge of SQL optimization and data modeling.

Details

• Location: Chennai, India.