Lead Data Engineer

Overview

Lead a team of engineers and hands-on design, build and operate the Service Layer platform powering enterprise Quote-to-Cash, customer lifecycle and finance integrations, using AWS, Kafka, Go and Python in a cloud-native, event-driven architecture.

What you'll do

• Own end-to-end technical delivery of key initiatives, ensuring quality, scalability, security and operational readiness.
• Lead and mentor a team of engineers, fostering technical excellence, knowledge sharing and continuous improvement.
• Drive architecture and design decisions for event-driven integrations, canonical data models, orchestration frameworks and platform capabilities.
• Define and enforce engineering standards covering secure-by-design principles, cloud best practices, data governance and non-functional requirements.
• Design, develop and operate real-time event-driven integration services using Golang and Python.
• Build and enhance the Service Layer platform that orchestrates Quote-to-Cash workflows across enterprise applications.
• Design and implement canonical data models, transformation frameworks, validation rules and integration contracts.
• Design and implement scalable Kafka-based event processing with retries, idempotency, sequencing, replay and fault recovery.
• Design and implement large-scale ETL, migration, reconciliation and data-quality solutions using AWS Glue, Step Functions, Snowflake, PostgreSQL and ClickHouse.
• Design and implement observability, monitoring, alerting and operational dashboards for end-to-end workflow visibility.
• Partner with Product Managers, Architects, Enterprise Data teams and other cross-functional stakeholders to define technical roadmaps and execution plans.
• Lead code reviews, design reviews, technical governance and engineering best-practice adoption across the team.
• Drive engineering standards covering software development lifecycle, CI/CD, security, observability, resiliency and cloud-native architecture patterns.

What you'll need

• Proven experience leading and mentoring engineering teams, including task planning, technical guidance, code reviews and delivery management.
• Strong experience driving architecture and design decisions for large-scale distributed systems and enterprise integration platforms.
• Strong hands-on experience in Golang and Python for building scalable backend services, event-driven integrations, orchestration workflows and data processing solutions.
• Strong experience designing and building cloud-native applications on AWS, including AWS Lambda, API Gateway, S3, RDS/Aurora PostgreSQL, DynamoDB, SQS, SNS, EventBridge, IAM, CloudWatch, VPC and related serverless architecture patterns.
• Experience managing technical delivery across multiple concurrent initiatives while balancing business priorities, technical debt and platform evolution.
• Strong experience designing and operating event-driven and asynchronous systems using Kafka, AWS MSK, RabbitMQ or equivalent messaging technologies.
• Strong understanding of distributed systems architecture, including scalability, resiliency, fault tolerance, idempotency, retry patterns, dead-letter queues, message replay and eventual consistency.
• Strong experience with PostgreSQL, ClickHouse, relational databases and NoSQL databases such as DynamoDB, including data modelling, query optimization and performance tuning.
• Hands-on experience designing and implementing RESTful APIs, integration services and enterprise system interfaces using secure API design principles.
• Experience building and enforcing canonical data models, data contracts, transformation layers and validation frameworks for enterprise integrations.
• Hands-on experience building and supporting high-availability, business-critical applications on AWS with strong focus on performance, reliability and operational supportability.
• Experience with CI/CD pipelines, automated testing, code quality and secure software delivery practices, using tools such as GitLab CI, Jenkins, SonarQube, Trivy and Semgrep.
• Experience with observability, monitoring and operational analytics, using CloudWatch, Grafana, Datadog, Prometheus, OpenTelemetry or equivalent platforms.
• Strong understanding of cloud security and secure-by-design principles, including IAM, OAuth2, JWT authentication, encryption at rest and in transit, Secrets Manager, vulnerability management, audit logging and compliance controls.
• Strong problem-solving skills with experience in production support, incident management, root cause analysis and performance optimization in large-scale distributed environments.
• Bachelor's degree in Computer Science, Engineering, Information Technology or related discipline.
• 10+ years of software engineering experience, with at least 5+ years focused on cloud-native, data engineering, integration or distributed platforms.
• 2+ years of experience leading engineering teams or acting as technical lead for large-scale enterprise initiatives.
• 5+ years building cloud-native solutions on AWS.
• 5+ years of experience working with Golang, Python, AWS services and Kafka in production environments.
• Proven experience designing, building and operating highly available enterprise integration and event-driven platforms.

Nice to have

• AWS certifications such as AWS Certified Data Engineer – Associate, AWS Certified Solutions Architect – Associate, AWS Certified Developer, or equivalent.
• Experience in Quote-to-Cash, CRM, ERP, Customer Lifecycle Management or Enterprise Data domains.

Details

• Location: Bangalore, India (Tower E, RMZ Infin).
• Career stage: Manager.