Senior Data Engineer

Overview

Design, build and maintain scalable data pipelines, integrations and analytical platforms supporting CTS's global analytics and reporting, working extensively with Azure Data Factory, Azure Databricks, Python, SQL and data warehouse structures.

What you'll do

• Design, build and optimize data pipelines using Azure Data Factory and Databricks (pyspark/spark SQL).
• Develop robust ETL/ELT processes to ingest, transform and validate large volumes of operational, telemetry, incident and transactional data.
• Implement scalable workflows leveraging Azure services such as Data Lake Storage, Azure Synapse, Azure SQL Databases, Key Vault, Logic Apps and Functions.
• Build efficient advanced SQL queries and stored procedures across Azure Synapse and Azure SQL Server.
• Collaborate with data analysts, engineers, performance assurance and operations teams to enable reliable, accurate datasets for reporting, KPIs and predictive analytics.
• Design and maintain data models, schemas, tables and metadata following COE architecture patterns.
• Implement strong data quality, validation and monitoring frameworks to ensure accuracy and reliability across global programs.
• Support junior engineers and uplift engineering capability within the COE.

What you'll need

• Bachelor's/master's degree in Computer Science, Data Analytics, Engineering, Mathematics or a related field.
• 5+ years of experience in data engineering roles.
• Strong expertise with Azure Data Factory (ADF) pipelines, Azure Databricks, mapping data flows and orchestrations.
• Advanced experience with Azure Databricks (pyspark, spark SQL, Delta Lake, notebooks, clusters).
• High proficiency in Python for ETL, automation and data transformation.
• Strong SQL skills across Azure Synapse and Azure SQL including query tuning and complex transformations.
• Solid understanding of data modelling, warehousing, star/snowflake schema, Medallion architecture and distributed data processing.
• Experience handling large-scale datasets and complex data domains.
• Familiarity with CI/CD practices, Git branching and DevOps pipelines.
• Experience with data governance, data cataloging, and managing structured/unstructured data in cloud environments.

Nice to have

• Experience with Azure Data Factory (ADF) pipelines, Azure Databricks, Databricks Workflows or Apache Airflow.
• Exposure to real-time/streaming data technologies (Event Hub, Kafka, Spark Streaming).
• Exposure to machine learning algorithms, feature engineering, model evaluation, hyperparameter tuning, LLM fundamentals and scikit-learn/XGBoost.
• Knowledge of deep learning, PyTorch/TensorFlow, MLflow, model deployment and MLOps practices.
• Experience integrating APIs, flat files and operational systems such as ServiceNow.
• Experience with machine learning pipelines, feature stores or model operationalization.
• Familiarity with Power BI dataset design and optimization.
• Understanding of multi-region cloud environments and enterprise architecture patterns.

Details

• Location: Hyderabad, Telangana.
• Worker type: Employee.
• Works in a fast-paced, global, multi-time zone team.