Expert Data & AI Engineer

Overview

Expert Data & AI Engineer in Hyderabad, responsible for designing and maintaining robust data pipelines and supporting integration and transformation projects.

What you'll do

• Provide technical recommendations and evaluate project feasibility and workloads.
• Design and implement data ingestion, transformation, and storage pipeline architectures.
• Develop modular code for data transformation and integration following modern development practices.
• Perform source-system data ingestion, including ETL when tables are unavailable on the platform.
• Maintain and upgrade pipelines, troubleshooting issues to ensure optimal performance.
• Implement automated data quality testing and validation for data integrity.
• Collaborate with other data and business stakeholders to deliver integrated, impactful solutions.

What you'll need

• 5 years of experience creating and maintaining data pipelines.
• Expert knowledge of dbt, SQL, Python, Spark (PySpark), Git, Azure DevOps, and Databricks.
• Experience with cloud engineering (AWS or Azure).
• Must deploy CI/CD pipelines.
• Strong problem-solving skills.
• Experience with deployment development and automation across environments.
• FAIR data principles, data governance, and scientific data standards experience.

Details

• Location: Hyderabad, India.
• Role includes team management, project oversight, and stakeholder engagement.