Senior Data Scheduling Infra Developer

Overview

Develop and operate a high-performance job scheduling and execution engine built on Apache Airflow. Enhance Airflow for large-scale data and AI workflow orchestration and contribute to the open-source community.

What you'll do

• Customize and enhance Apache Airflow source code.
• Develop features and improve Airflow functionality and performance.
• Contribute patches, enhancements, and fixes to the Airflow community.
• Implement secure, scalable, and high-performance Airflow architectures.
• Develop and optimize complex Airflow DAGs and Operators.
• Track developments in Airflow and related technologies.

What you'll need

• 5+ years of software development experience focused on Python.
• 2+ years of hands-on Apache Airflow experience.
• Experience developing Airflow DAGs and Operators.
• Understanding of cloud platforms and Docker and Kubernetes.
• Experience with CI/CD tools and practices.
• Bachelor's degree in Computer Science, Information Technology, or a related field.
• Experience delivering production-grade software using AI coding tools.

Nice to have

• Advanced degree.
• Deep understanding of Apache Airflow architecture and internals.
• Apache Airflow or similar open-source contribution experience.
• Experience with Hadoop, Spark, Kafka, or Flink.

Details

• Location: Bengaluru, India.