Databricks Architect

Overview

The Databricks Architect will lead the design, implementation, governance, and optimization of enterprise-scale data and AI platforms on Databricks. This is a full-time, remote position.

What you'll do

• Define and own the enterprise Databricks architecture strategy, ensuring scalability, security, reliability, and alignment with business and AI objectives.
• Design modern data platforms leveraging Databricks Lakehouse, Delta Lake, Unity Catalog, and AWS services.
• Lead the architecture and modernization of data warehouses, data lakes, and analytical platforms into scalable Lakehouse architectures.
• Establish enterprise standards, frameworks, and best practices for data ingestion, transformation, orchestration, governance, and consumption.
• Design and implement secure, high-performance batch and real-time data processing solutions.
• Develop reference architectures for data engineering, advanced analytics, machine learning, and generative AI workloads.
• Define data governance, lineage, metadata management, security, and compliance standards across the data platform.
• Architect scalable and reusable data models that support reporting, analytics, operational intelligence, and AI use cases.
• Guide implementation teams on platform setup, workspace design, networking, access controls, monitoring, and operational readiness.
• Drive platform optimization initiatives related to compute utilization, workload management, storage performance, and cost efficiency.
• Lead architectural reviews and provide oversight for critical data and AI initiatives.
• Collaborate with business stakeholders, product owners, data engineers, ML engineers, and cloud teams to translate requirements into technical solutions.
• Establish observability frameworks for monitoring platform health, performance, data quality, and usage patterns.
• Evaluate emerging Databricks capabilities and recommend adoption strategies that enhance business value.
• Develop migration strategies and execution roadmaps for legacy data platforms transitioning to Databricks.
• Define disaster recovery, business continuity, backup, and resilience requirements for enterprise data platforms.
• Drive architecture governance through design reviews, standards enforcement, and continuous improvement initiatives.
• Mentor engineering teams and provide technical leadership on complex architecture decisions.
• Deliver executive-level architecture documentation, roadmaps, solution blueprints, and technology recommendations.
• Architect solutions across AWS and Azure environments and integrate Databricks with AWS and Azure native services to support enterprise analytics, data engineering, and AI use cases.

What you'll need

• Bachelor's degree or higher in Computer Science, Information Technology, Engineering, Data Science, or a related field.
• 10+ years of experience in data engineering, data architecture, cloud architecture, or enterprise technology leadership roles.
• 5+ years of hands-on experience with Databricks and modern cloud-based data platforms.
• Extensive experience designing and implementing enterprise-scale data lake, data warehouse, and Lakehouse architectures.
• Strong expertise in Databricks, Delta Lake, Unity Catalog, Workflows, and Databricks SQL.
• Hands-on experience with AWS services such as S3, IAM, Glue, Lambda, EC2, RDS, Redshift, CloudWatch, and networking services.
• Experience designing data pipelines using Spark, PySpark, SQL, and modern orchestration frameworks.
• Deep understanding of data governance, security, metadata management, cataloging, lineage, and compliance requirements.
• Experience with CI/CD, Infrastructure as Code, DevOps, and platform automation practices.
• Strong knowledge of analytics, machine learning, and AI data platform requirements.
• Experience establishing enterprise architecture standards, patterns, and best practices.
• Proven ability to design highly available, scalable, and cost-efficient cloud architectures.
• Strong stakeholder management skills with experience engaging executive leadership, business stakeholders, and technical teams.
• Ability to evaluate trade-offs and make architecture decisions in complex and ambiguous environments.
• Strong communication and presentation skills with the ability to influence technical and non-technical audiences.
• Proven leadership experience driving large-scale transformation and modernization initiatives.
• Hands-on experience with both AWS and Azure cloud platforms, including architecture design, security, networking, storage, governance, and data services.
• Strong understanding of multi-cloud data platform strategies and the ability to architect solutions across AWS and Azure environments.

Nice to have

• Knowledge of Databricks Genie, AI/BI capabilities, and natural language-driven analytics solutions.
• Databricks Certified Data Engineer Professional certification.
• Databricks Certified Machine Learning Professional certification.
• Databricks Certified Platform Architect certification.
• AWS Solutions Architect Associate certification.

Details

• Employment type: Full time.
• Work mode: Remote.
• Location: India - Bengaluru.
• The role involves sedentary work that includes sitting or remaining stationary most of the time, with occasional need to move around the office to attend meetings.
• The role involves conducting repetitive tasks on a computer using a mouse, keyboard, and monitor.
• Applicants must be able to safely and successfully perform the essential job functions.