Data Architect

Overview

Design and implement scalable, secure data architectures and Databricks-based pipelines. Build data integrations, governance capabilities, dashboards, and GenAI-ready datasets.

What you'll do

• Develop scalable ETL/ELT pipelines using Databricks, Delta Lake, Auto Loader, and DLT.
• Build Python and PySpark transformation, validation, and debugging solutions.
• Configure Unity Catalog access, auditing, PII masking, external locations, volumes, and lineage.
• Implement secure DLT pipelines with expectations and Bronze/Silver/Gold layering.
• Integrate BI tools and build Databricks SQL dashboards for KPIs and pipeline monitoring.
• Deploy, troubleshoot, and optimize pipelines using Asset Bundles, CI/CD, Photon, and serverless compute.
• Prepare GenAI datasets using embeddings, Vector Search, and Feature Store with MLflow.

What you'll need

• At least 7.5 years of experience with Databricks Unified Data Analytics Platform.
• 15 years of full-time education.
• Proficiency in Databricks, Python, and PySpark.
• Hands-on experience with at least one of AWS, GCP, or Azure.
• Experience with data modeling, data warehousing, cloud data services, and performance optimization.
• Understanding of real-time streaming, serverless computing, and secure data access.

Nice to have

• Databricks certification or similar certification.
• Machine learning concepts and popular ML libraries.
• Big data technologies such as Spark, Hadoop, Hive, or Kafka.
• Apache Airflow, CI/CD, DevOps, ETL tools, or DBT experience.

Details

• Location: Bengaluru.