Senior Specialist, Data Engineering

Overview

Lead the design and delivery of scalable data engineering solutions using Databricks, AWS, IDMC, and advanced analytics technologies. The role supports Salesforce and Veeva CRM integrations and includes technical leadership, architecture, and mentoring responsibilities.

What you'll do

• Lead complex batch and real-time data pipeline architecture, construction, and maintenance.
• Optimize Databricks and AWS processing pipelines for large-scale datasets.
• Define, document, and promote data engineering standards and best practices.
• Deliver data analytics initiatives from ingestion through reporting.
• Translate stakeholder goals into technical architectures and data solutions.
• Prototype emerging pipeline patterns and lead technology proofs of concept.
• Mentor junior engineers and maintain accountability for delivery quality and timelines.

What you'll need

• 5–8 years of experience.
• Expertise in PySpark, Python, advanced SQL, distributed computing, and query optimization.
• Experience designing high-throughput ETL/ELT pipelines with IDMC.
• Deep knowledge of Databricks optimization, Unity Catalog, and Delta Lake.
• Experience with AWS, Terraform, Lambda, S3, IAM, and Step Functions.
• Expertise in data modeling, data warehousing, CI/CD, and testing automation.
• Experience writing architecture documents, defining standards, and mentoring engineers.

Nice to have

• Communication skills, including strong writing and storytelling.
• Experience with Salesforce or Veeva CRM replication pipelines.

Details

• Location: Hyderabad, Telangana, India.
• Work arrangement: Hybrid.
• Employment status: Regular.
• Travel: No travel required.