Lead Data Engineer

Overview

Lead Data Engineer to join the billing transformation program, building the pipelines that turn raw telemetry usage data into rated, validated, invoice-ready billing data for SAP invoicing.

What you'll do

• Source and ingest telemetry data through batch and event-driven data ingestion pipelines.
• Build curated usage datasets in Snowflake.
• Implement rating logic and produce billing-ready datasets.
• Generate the SAP invoice feed.
• Implement the data quality and reconciliation framework.
• Build the exception and error-handling process.
• Build and maintain CI/CD pipelines for data engineering deployments.
• Set up monitoring, alerting, and operational dashboards.
• Write technical documentation and runbooks.

What you'll need

• Strong hands-on experience in data engineering, ETL/ELT development, and large-scale data pipeline design.
• Experience with AWS Glue, PySpark, Python, SQL, and cloud-based data processing.
• Strong experience working with Snowflake as a data warehouse or enterprise data platform.
• Good understanding of usage-based billing, rating engines, invoice feeds, or financial data processing.
• Experience building data quality checks, reconciliation frameworks, audit controls, and exception handling processes.
• Hands-on experience with CI/CD tools such as GitHub Actions, GitLab CI, Jenkins, AWS Code Pipeline, or similar.
• Strong SQL skills for data analysis, validation, troubleshooting, and performance optimization.
• Experience working with batch and event-driven data ingestion patterns.
• Understanding of data modelling, partitioning, performance tuning, and cost optimization.
• Strong analytical skills and attention to detail, especially for financial accuracy.
• Ability to collaborate with cross-functional teams including Finance, Engineering, and Analytics.
• Good communication skills to explain defects, logic, and test outcomes clearly.
• High ownership and adaptability, with willingness to take on both QA and minor development tasks.

Details

• Location: Hyderabad, India (CapitaLand).
• Hybrid working; the role is Blended, requiring office attendance at least three days per week.
• Career stage: Senior Associate.