Data Engineer

Overview

Design, develop and optimize scalable data pipelines and cloud-native data solutions on Google Cloud Platform, including RAG data pipelines, vector infrastructure, semantic layers and CI/CD for data infrastructure.

What you'll do

• Architect and scale pipelines for Retrieval-Augmented Generation (RAG), transforming unstructured IT logs and documentation into optimized vector embeddings.
• Own the health and performance of vector databases (e.g., Pinecone, Milvus, or Weaviate), ensuring sub-second retrieval speeds for agentic reasoning loops.
• Engineer semantic layers and knowledge graphs that give agents the context needed to navigate complex infrastructure.
• Build automated data guardrails to detect noise, bias, or PII (Personally Identifiable Information) before data reaches the model.
• Identify and resolve data quality issues at the source, bridging raw, messy data sources and technical AI work.
• Build, deploy and maintain CI/CD pipelines for data infrastructure, keeping the context window fresh and reliable.

What you'll need

• Overall 6+ years of experience in IT.
• Expertise in data mining, data storage and Extract-Transform-Load (ETL) processes.
• Experience in data pipeline development and tooling, e.g., Glue, Databricks, Synapse, or Dataproc.
• Experience with both relational and NoSQL databases, PostgreSQL, DB2, MongoDB.
• Excellent problem-solving, analytical, and critical thinking skills.
• Ability to manage multiple projects simultaneously, while maintaining a high level of attention to detail.
• Ability to communicate with both technical and non-technical colleagues, to derive and translate technical requirements from business needs.

Nice to have

• Experience working as a Data Engineer and/or in cloud modernization.
• Experience in Data Modelling, to create conceptual model of how data is connected and how it will be used in business processes.
• Professional certification, e.g. Open Certified Technical Specialist with Data Engineering Specialization.
• Cloud platform certification, e.g. AWS Certified Data Analytics – Specialty, Elastic Certified Engineer, Google Cloud Professional Data Engineer, or Microsoft Certified: Azure Data Engineer Associate.
• Understanding of social coding and Integrated Development Environments, e.g. GitHub and Visual Studio.
• Degree in a scientific discipline, such as Computer Science, Software Engineering, or Information Technology.
• Skills: DBT, GCP, BigQuery, Terraform, CI/CD, DevOps, SQL, Spark, Apache Beam, Hive, Cloud Run, Iceberg, Hudi, Avro, Parquet, Data Mesh, Data Modeling, ETL, ELT, Data Pipeline, Google Cloud.

Details

• Location: Bangalore, Karnataka, India.
• Hybrid-friendly work environment.