Data Engineer

Overview

Hands-on engineering role in AI LLM Technology Architecture, designing, building, integrating, testing and operationalizing enterprise-grade LLM, GenAI and agentic AI components across active client engagements, with platform-specific engineering on Databricks.

What you'll do

• Design and build LLM application components including prompts, tools, agents, orchestration flows, memory/context handling, retrieval pipelines and evaluation harnesses.
• Build data-grounded agentic applications on the lakehouse: implement RAG with Delta tables, Vector Search and governed features; use MLflow for tracing, evaluation and model lifecycle; deploy agents or models with Model Serving; enforce governance through Unity Catalog.
• Implement data ingestion, parsing, chunking, enrichment, embeddings, vector search and retrieval workflows for structured and unstructured enterprise content.
• Engineer safety and control components including PII detection/redaction, prompt-injection defenses, content filters, guardrails, authentication, authorization, lineage and audit logging.
• Collaborate with architects, data engineers, product owners and security stakeholders to convert solution designs into tested, observable and maintainable software components.
• Maintain technical artifacts such as component designs, integration specifications, deployment runbooks, evaluation results and reusable engineering patterns.

What you'll need

• Bachelor's degree in Computer Science, Computer Engineering, Data Science, AI/ML, Information Technology or a related engineering discipline.
• Hands-on coding experience in Python and strong understanding of APIs, distributed systems, CI/CD, testing, observability and secure SDLC practices.
• Experience delivering AI/ML or data products in at least one industry domain such as financial services, healthcare, manufacturing, retail, telecom or life sciences.
• Minimum 7.5 year(s) of experience is required.
• Must have skills: Databricks Unified Data Analytics Platform.
• Hands-on experience with Databricks Mosaic AI, Model Serving, Agent Framework, MLflow tracing/evaluation, Vector Search, Unity Catalog, Delta Lake, Lakehouse Monitoring, Feature Store, Databricks Workflows, Jobs, notebooks and Model Training.
• Strong understanding of LLM application architecture patterns including RAG, function/tool calling, agent orchestration, model invocation, prompt engineering, embeddings, vector databases and evaluation metrics.
• Ability to implement traditional ML and GenAI components across ingestion, feature/data preparation, model integration, deployment, monitoring and continuous improvement.
• Practical knowledge of security, privacy, governance, performance, scalability, reliability and cost controls for production AI systems.
• Experience with Git-based development, automated testing, CI/CD pipelines, infrastructure-as-code and agile delivery in client-facing environments.
• 15 years full time education.

Nice to have

• Good to have skills: Large Language Models (LLMs).
• Databricks Machine Learning, Data Engineer or Generative AI certification; experience with Spark/PySpark, Delta Live Tables, Unity Catalog governance, LangGraph/LangChain, model fine-tuning and lakehouse cost/performance optimization.
• Exposure to open-source frameworks such as LangChain, LangGraph, LlamaIndex, Haystack, MLflow, FastAPI, Docker and Kubernetes.
• Experience with Responsible AI, model risk management, synthetic data generation, human-in-the-loop review, A/B testing and GenAI cost optimization.

Details

• Location: Chennai.