Senior Data Engineer

Overview

Work as a hands-on, highly autonomous Senior Data Engineer designing, building, and operating the Service Layer platform that powers enterprise Quote-to-Cash, customer lifecycle, and finance integrations. Build and evolve a cloud-native, event-driven integration platform using AWS, Kafka, Go, and Python.

What you'll do

• Design, develop, and operate real-time event-driven integration services using Golang and Python.
• Build and enhance the Service Layer platform that orchestrates Quote-to-Cash workflows across enterprise applications.
• Design and implement canonical data models, transformation frameworks, validation rules, and integration contracts.
• Develop scalable Kafka-based event processing solutions with retries, idempotency, sequencing, replay, and fault recovery.
• Build secure REST APIs through AWS API Gateway and integrate with SaaS platforms and enterprise applications.
• Build and support large-scale ETL, migration, reconciliation, and data-quality solutions using AWS Glue, Step Functions, Snowflake, PostgreSQL, and ClickHouse.
• Design and implement observability, monitoring, alerting, and operational dashboards for end-to-end workflow visibility and supportability.
• Define and enforce engineering standards, secure-by-design principles, cloud best practices, data governance, and non-functional requirements for resilience, scalability, and supportability.
• Collaborate with distributed engineering teams and cross-functional stakeholders to translate business requirements into scalable data solutions delivered within agreed timelines.
• Participate in architecture reviews, production support, root-cause analysis, performance optimization, and operational readiness activities.

What you'll need

• Bachelor's degree in Computer Science, Engineering, Information Technology, or related discipline.
• 7+ years of software engineering experience.
• At least 3+ years focused on data engineering and distributed data platforms.
• 3+ years building cloud-native solutions on AWS.
• 3+ years of experience working with Golang, Python, AWS Service, and Kafka in production environments.
• Strong hands-on experience in Golang and Python for scalable backend services, event-driven integrations, orchestration workflows, and data processing solutions.
• Strong experience designing and building cloud-native applications on AWS, including AWS Lambda, API Gateway, S3, RDS/Aurora PostgreSQL, DynamoDB, SQS, SNS, EventBridge, IAM, CloudWatch, VPC, and related serverless architecture patterns.
• Strong experience designing and operating event-driven and asynchronous systems using Kafka, AWS MSK, RabbitMQ, or equivalent messaging technologies.
• Strong understanding of distributed systems architecture, including scalability, resiliency, fault tolerance, idempotency, retry patterns, dead-letter queues, message replay, and eventual consistency.
• Strong experience with PostgreSQL, ClickHouse, relational databases, and NoSQL databases such as DynamoDB, including data modelling, query optimization, and performance tuning.
• Hands-on experience designing and implementing RESTful APIs, integration services, and enterprise system interfaces using secure API design principles.
• Experience building and enforcing canonical data models, data contracts, transformation layers, and validation frameworks for enterprise integrations.
• Hands-on experience building and supporting high-availability, business-critical applications on AWS with strong focus on performance, reliability, and operational supportability.
• Experience with CI/CD pipelines, automated testing, code quality, and secure software delivery practices, using tools such as GitLab CI, Jenkins, SonarQube, Trivy, Semgrep etc.
• Experience with observability, monitoring, and operational analytics, using CloudWatch, Grafana, Datadog, Prometheus, OpenTelemetry, or equivalent platforms.
• Strong understanding of cloud security and secure-by-design principles, including IAM, OAuth2, JWT authentication, encryption at rest and in transit, Secrets Manager, vulnerability management, audit logging, and compliance controls.
• Strong problem-solving skills with experience in production support, incident management, root cause analysis, and performance optimization within large-scale distributed environments.
• Experience designing and supporting highly available enterprise integration platforms.

Nice to have

• AWS certifications such as AWS Certified Data Engineer – Associate, AWS Certified Solutions Architect – Associate, AWS Certified Developer, or equivalent.
• Experience in Quote-to-Cash, CRM, ERP, Customer Lifecycle Management, or Enterprise Data domains.

Details

• Location: Bangalore, India.
• Career Stage: Senior Associate.