Senior Data Engineer

Overview

Own and operate Illumina's enterprise Databricks Lakehouse platform, ensuring reliability, security, scalability, automation, governance and cost efficiency. Build reusable platform patterns, support AI/ML workloads and mentor engineers in the India team.

What you'll do

• Own Databricks workspace and account architecture, administration and environment promotion.
• Build infrastructure-as-code, CI/CD and observability automation.
• Manage platform reliability, SLOs, incidents, capacity and disaster recovery.
• Manage Databricks consumption, cost optimisation, budgets and chargeback.
• Implement security and governance controls, including identity, access, auditing and PII handling.
• Enable AI/ML capabilities including MLflow, Model Serving and vector search.
• Mentor engineers and own the platform roadmap and engineering standards.

What you'll need

• 3–5 years of professional data platform or data engineering experience.
• At least 2 years administering and operating Databricks in production.
• Infrastructure-as-code and CI/CD experience using Terraform or equivalent.
• Strong Python, PySpark and advanced SQL skills.
• Experience with monitoring, incident management, runbooks and platform operations.
• Experience with Databricks FinOps, security, compliance and governance controls.
• Bachelor's degree in a relevant field or equivalent demonstrable experience.

Nice to have

• Databricks Certified Data Engineer Professional or AWS Certified Solutions Architect certification.
• Experience with compliance-driven platforms, mentoring engineers or migrating an existing platform.
• Experience working with distributed global stakeholders.

Details

• Location: Bengaluru, India.