Data Engineer

Overview

Data Engineer to design, build, and maintain data pipelines for digital twin platforms, real-time operational systems, and AI/ML workloads.

What you'll do

• Design, develop, and maintain scalable data pipelines using Databricks, PySpark, and Delta Lake.
• Build real-time and batch ingestion pipelines from operational systems using Kafka.
• Transform raw operational data into governed, high-quality datasets for analytics and digital twin platforms.
• Integrate Kafka event streams with Databricks for real-time operational state updates.
• Implement data quality checks using Delta Live Tables expectations.
• Ensure governance compliance via Unity Catalog (lineage, access control, metadata).
• Optimize pipeline performance, reliability, and cost efficiency; write tested, documented code.

What you'll need

• 7+ years of hands-on data engineering experience.
• Experience building production-grade data pipelines.
• Experience with Delta Live Tables for declarative pipeline development.
• Experience working in agile, cross-functional teams.
• Familiarity with time-series data patterns and operational data modelling.

Nice to have

• Experience building pipelines for digital twin or simulation platforms.
• Familiarity with operational state modeling for real-time systems.
• Exposure to physics-informed or time-series ML feature engineering.
• Exposure to industrial domains such as Manufacturing, Logistics, or Transportation.

Details

• Location: Hyderabad, India (office).