Staff Data Engineer (8-10 years' exp - Java/Python, Scala, Spark, Hadoop)

Overview

Staff Data Engineer to design, implement, and improve secure, scalable data engineering solutions in the fintech domain.

What you'll do

• Modernize data platforms by transitioning from SQL Server to Hadoop/Databricks.
• Build hybrid data pipelines bridging legacy and modern ecosystems for near real-time analytics and AI.
• Implement distributed data pipelines using Spark, Kafka, Airflow, and Delta Lake.
• Optimize SQL Server models and design modern equivalents in Hadoop and Spark.
• Develop ETL/ELT frameworks for transactional, behavioral, and unstructured data processing.
• Support AI-driven insights by enabling RAG pipelines and vector database integrations.
• Provide production guidance for SQL Server warehouses, including tuning, replication, encryption, and HA.

What you'll need

• 8+ years of hands-on data engineering experience in enterprise-scale warehousing and pipelines.
• Proven success modernizing SQL Server-based data warehouses in high-SLA environments.
• Production-level experience architecting Hadoop, Spark, and Databricks pipelines.
• Strong programming proficiency in Python, Java, or Scala.
• Hands-on experience with AWS or Azure services including Glue, Synapse, Redshift, Delta Lake, and S3.
• Familiarity with Kafka, Airflow, Kubernetes, and containerized data services.
• Experience with SQL optimization, debugging, and production troubleshooting.

Details

• Location: Bengaluru, India.
• On-site expectation: at least 3 days in office (exact days confirmed by the Hiring Manager).