Senior Associate – Data Engineering, Advisory

Overview

Lead data engineering engagements as a Senior Associate, designing and developing scalable data solutions, pipelines, architectures and platforms for clients. The role focuses on data services, data architecture and data platforms, with hands-on expertise in Spark, PySpark and Python.

What you'll do

• Design, develop, and maintain scalable data pipelines and architectures using Spark, PySpark, and Python.
• Lead and manage a team of data engineers, providing technical guidance, mentorship, and performance management.
• Collaborate with cross-functional teams to understand data requirements and deliver high-quality data solutions.
• Ensure data quality, integrity, and security across all data engineering projects.
• Implement best practices for data engineering, including code reviews, testing, and documentation.
• Stay up-to-date with the latest industry trends and technologies in data engineering and cloud computing.
• Work closely with stakeholders to understand business needs and translate them into technical requirements.
• Manage project timelines, resources, and budgets to ensure successful delivery of data engineering projects.

What you'll need

• 5-8 years of experience specializing in data services, data architecture, and data platforms.
• Strong experience with data engineering technologies, particularly Spark, PySpark, and Python.
• Proven track record of designing and building scalable data pipelines and architectures.
• Hands-on experience with data processing and transformation.
• Strong understanding of data warehousing concepts and ETL processes.
• Excellent problem-solving skills and the ability to troubleshoot complex technical issues.
• Strong leadership and team management skills.
• Effective communication skills, both written and verbal.
• Spark 3.0 certification is mandatory.
• B.Tech, M.Tech, MCA/MBA; degrees/fields of study required include Master of Business Administration and Bachelor of Engineering.

Nice to have

• Databricks Advanced/Professional Architect certification.
• Experience with cloud-native data engineering on platforms such as Databricks and Azure/AWS.
• Familiarity with cloud services related to data storage, processing, and analytics, such as BigQuery, Redshift, S3, and EMR.
• Knowledge of containerization and orchestration technologies, such as Docker and Kubernetes.
• Experience with data visualization tools and techniques.
• Familiarity with machine learning and data science concepts.
• Proven experience in data engineering or related roles.
• Industry certifications related to data engineering or cloud platforms are a plus.

Details

• Location: Bengaluru Millenia, Bangalore.
• Employment type: Full Time.
• Travel requirements: Not specified.
• Work visa sponsorship: No.
• Government clearance required: No.