Senior Software Engineer

Overview

Design, build and operate scalable data pipelines and analytical solutions on AWS and Snowflake, integrating data from enterprise applications, APIs, databases and files into reliable datasets for analytics, reporting and AI/ML use cases.

What you'll do

• Design and maintain batch and incremental data ingestion pipelines from APIs, relational databases, SaaS applications and file sources.
• Build AWS ingestion solutions using services such as Amazon S3, AWS Glue, Lambda, IAM and Step Functions.
• Develop Snowflake data models and transformation pipelines using SQL, Python, dbt or equivalent tools.
• Implement Bronze, Silver and Gold layers within a medallion architecture, with clear data contracts and lineage.
• Process structured and semi-structured data, including JSON, CSV and Parquet.
• Implement change data capture, incremental loading, deduplication, schema evolution and historical data management.
• Integrate datasets across systems using consistent business keys, reference mappings and transformation rules.
• Build data quality checks, reconciliation controls, monitoring, alerting and recovery procedures.
• Optimize Snowflake queries, warehouse usage and storage to improve performance and control costs.
• Implement access controls, encryption, secrets management and data protection requirements.
• Establish automated testing, CI/CD and repeatable deployment practices for data pipelines.
• Troubleshoot production issues, perform root cause analysis and improve pipeline reliability.
• Collaborate with analysts, data scientists, architects and business stakeholders to translate requirements into technical solutions.
• Maintain technical documentation and provide engineering guidance through design and code reviews.

What you'll need

• 8+ years of professional experience in data engineering, data warehousing, ETL/ELT development or related roles.
• Strong hands-on experience with AWS and Snowflake in production environments.
• Advanced SQL skills, including complex transformations, analytical functions and query optimization.
• Strong Python skills for API integrations, data processing, automation and error handling.
• Experience building ingestion pipelines with pagination, authentication, rate limiting, retries and checkpointing.
• Experience with Snowflake capabilities such as stages, storage integrations, COPY INTO, Snowpipe, Streams and Tasks.
• Experience with transformation frameworks such as dbt or equivalent SQL-based tooling.
• Experience with orchestration tools such as Apache Airflow, AWS Step Functions or equivalent platforms.
• Strong understanding of dimensional modeling, slowly changing dimensions and incremental processing.
• Experience implementing data quality, observability, lineage and operational support practices.
• Proficiency with Git, code reviews and CI/CD.
• Strong problem-solving, communication and stakeholder collaboration skills.

Nice to have

• Experience preparing and managing datasets for AI/ML, including feature engineering and training and inference pipelines.
• Familiarity with MLOps, including experiment tracking, model versioning, deployment, monitoring and retraining workflows.
• Experience with tools such as Amazon SageMaker, MLflow, Snowpark or equivalent platforms.
• Familiarity with model drift, data drift and reproducibility.
• Experience supporting generative AI applications, embeddings, vector search or retrieval-augmented generation.
• Experience with streaming technologies such as Amazon Kinesis or Apache Kafka.
• Experience with infrastructure as code using Terraform or AWS CloudFormation.
• AWS, Snowflake or relevant data engineering certifications.

Details

• Location: Pune, India.
• Work from home.
• Competitive salary and opportunity to earn an annual bonus.