Senior Software Developer

Overview

Senior Data Engineer to design, build, and evolve scalable data pipelines, data models, and data products on Tecsys's analytics platform, focused on batch-first ETL/ELT systems on Databricks and Spark that produce AI-consumable datasets supporting Search, Recommendations, Marketing, and Supply Chain analytics.

What you'll do

• Design, build, and maintain scalable ETL/ELT pipelines that ingest and transform structured, semi-structured, and unstructured data.
• Develop high-fidelity data pipelines on Databricks/Spark optimized for reliability, cost, performance, and data freshness.
• Build curated datasets, embeddings-ready data, feature layers, and semantic abstractions that are AI/ML-consumable for downstream systems.
• Implement ingestion, transformation, and serving layers across Data Lake / Lakehouse architectures with a focus on efficient retrieval and contextual usability.
• Develop and maintain robust data models including fact/dimension models, SCDs, wide tables, and CDC pipelines.
• Apply data versioning, incremental processing, partitioning, and clustering strategies to ensure consistency, reproducibility, and cost efficiency.
• Contribute to architectural decisions and trade-offs across storage, compute, and orchestration layers within the analytics platform.
• Help define and uphold data modeling standards, data contracts, and quality frameworks across teams.
• Prepare high-quality datasets for ML model consumption, feature engineering workflows, and predictive/forecasting use cases.
• Contribute to a unified semantic layer that standardizes metrics, reusable definitions, and improves data access patterns.
• Partner with Data Science teams to operationalize feature pipelines and support model training, serving, and monitoring.
• Implement data quality checks, contracts, and observability to ensure SLA/SLO adherence across pipelines.
• Work with metadata, lineage, and data discovery frameworks to improve transparency, governance, and trust in data.
• Drive improvements in pipeline reliability, monitoring, and incident response across the data ecosystem.
• Partner cross-functionally with Product, Analytics, and Data Science to translate ambiguous business problems into reusable data assets.
• Mentor junior engineers, review code/designs, and raise the bar for engineering quality and best practices.
• Communicate technical decisions, trade-offs, and system designs clearly to both technical and non-technical stakeholders.

What you'll need

• Advanced proficiency in SQL and Python, with strong focus on query optimization, cost efficiency, and large-scale data processing.
• Hands-on experience with Databricks, Apache Spark, and distributed processing frameworks.
• Strong experience building ETL/ELT pipelines and workflow orchestration (Airflow, Dagster, or similar).
• Solid understanding of Data Lake / Lakehouse architectures, storage formats (Parquet, Delta/Iceberg), partitioning, clustering, and performance tuning.
• Deep expertise in data modeling: fact/dimension models, SCDs, wide tables, CDC, data versioning, and incremental processing.
• Experience with event-driven and streaming architectures (Kafka, Pub/Sub, Kinesis) applied pragmatically alongside batch where needed.
• Hands-on experience with at least one major cloud platform (AWS, Azure, or GCP) and cloud data warehouses (Snowflake, BigQuery, Redshift).
• Familiarity with Docker, Kubernetes, CI/CD pipelines, and infrastructure-as-code for data systems.
• Experience building production-grade data systems with strong data quality, observability, and SLA/SLO practices.
• Familiarity with metadata management, data lineage, and data discovery tools (e.g., DataHub, OpenMetadata, Amundsen).
• Proven ability to translate ambiguous business requirements into scalable data models and systems.
• Ownership mindset: drives features end-to-end with strong communication and collaboration skills.
• Bachelor's or Master's in Computer Science, Engineering, or related field, or equivalent practical experience.
• 6–8+ years of experience in Data Engineering, building and operating production-grade data platforms at scale.
• Demonstrated experience delivering data solutions across analytics, ML, or product-facing domains.

Nice to have

• Experience enabling AI/ML use cases from a data perspective: preparing datasets, feature stores, embeddings, or semantic/metric layers (dbt, Cube, LookML).
• Exposure to BI tools (Power BI, Tableau, Looker) and KPI modeling for business stakeholders.
• Experience in Supply Chain, Logistics, or Healthcare supply chain analytics.
• Familiarity with domains such as Search, Recommendations, or Marketing analytics.

Details

• Location: Bengaluru, Karnataka, India; role requires being based in Bengaluru.