Senior Azure Cloud Platform Engineer

Overview

Senior Azure Cloud Platform Engineer role providing technical leadership and architecting Azure-based data solutions, including Azure Data Lake, databases, Databricks and Azure Data Factory.

What you'll do

• Provide technical leadership and mentorship to cross-functional teams (cloud engineers, data engineers, business stakeholders) to architect robust Azure-based data solutions.
• Ensure alignment of cloud solutions with business objectives, considering current and future growth.
• Stay updated with the latest Azure innovations, trends, and best practices to recommend the best solutions for complex business needs.
• Design and deploy scalable and high-performance Azure Data Lake storage solutions to handle large volumes of unstructured and structured data.
• Architect, configure, and manage Azure SQL Database, Cosmos DB, and other database offerings to ensure high availability, disaster recovery, and optimized performance.
• Manage, configure, and administer Azure Databricks environments, ensuring optimal performance, scalability, and security of data processing workloads.
• Collaborate with data scientists, data engineers, and analysts to integrate Azure Databricks with other Azure services (e.g., Azure Data Lake, Azure Synapse) to build scalable data processing and analytics solutions.
• Implement best practices for managing Databricks clusters, notebooks, libraries, and jobs to ensure efficiency and cost optimization.
• Perform regular monitoring, troubleshooting, and performance tuning of Databricks workloads, ensuring high reliability and uptime.
• Develop and implement comprehensive strategies for migrating on-premises data and applications to Azure with minimal disruption to business operations.
• Use Azure-native tools (e.g., Azure Migrate, Data Migration Assistant) to facilitate seamless migration of data assets to the cloud.
• Design, implement, and maintain efficient data pipelines using Azure Data Factory (ADF) to orchestrate and automate workflows across various sources and destinations.
• Optimize pipeline performance, manage dependencies, and ensure seamless data movement across hybrid cloud environments.
• Administer and optimize Azure Database offerings such as Azure SQL Database, SQL Managed Instances, and other relational/non-relational databases.
• Design high-availability, fault-tolerant architectures, and ensure efficient data replication, backup, and recovery.
• Identify and address performance bottlenecks within Azure-based data solutions, applying performance tuning strategies to maximize system efficiency.
• Ensure systems are optimized for cost efficiency, scalability, and reliability by monitoring usage patterns and implementing proactive optimization strategies.
• Implement strong security measures across Azure data solutions to ensure confidentiality, integrity, and availability of data.
• Leverage Azure's security features, including encryption, role-based access control (RBAC), firewalls, and auditing tools to meet compliance requirements (e.g., GDPR, HIPAA, SOC 2).
• Conduct regular security assessments and work with teams to enforce security best practices across all cloud solutions.
• Engage with stakeholders from business units, IT, and data engineering to understand business requirements, design appropriate solutions, and ensure smooth project delivery.
• Present project updates, technical challenges, and solutions to executive leadership and other stakeholders, ensuring transparency and alignment with organizational goals.
• Facilitate collaborative decision-making and knowledge sharing among cross-functional teams to achieve optimal solutions.
• Lead the design and implementation of Infrastructure as Code (IaC) solutions using tools like Azure Resource Manager (ARM) templates, Terraform, or Azure Bicep to automate infrastructure provisioning and management.
• Ensure that IaC practices are embedded in the development lifecycle for consistent, repeatable, and scalable cloud deployments.

What you'll need

• 6+ years of experience in cloud architecture.
• At least 4 years of experience focused on Azure cloud solutions and architecture design.
• Expertise in Azure Data Lake, Azure SQL Database, Azure Cosmos DB, Azure SQL Managed Instance, and related Azure data services.
• Strong proficiency in Azure Data Factory (ADF) for data pipeline orchestration and ETL process design.
• Databricks administration and optimization experience, including cluster management, job execution, and integration with Azure services.
• Solid experience with Infrastructure as Code (IaC) using tools like Terraform, ARM templates, or Azure Bicep.
• Deep understanding of cloud security best practices, including access control, data encryption, and compliance standards.
• Experience with CI/CD pipelines and automation tools.
• Leadership experience, with a track record of successfully driving technical initiatives to completion.

Nice to have

• Familiarity with Azure Databricks and other advanced data processing services.

Details

• Location: Hyderabad, India.
• Work mode: Hybrid.