07.Data Engineer

Overview

Data Engineer to build and scale the data pipeline platform for PayPay India’s payment system and customer experience.

What you'll do

• Create and manage data ingestion pipelines using Databricks, Airflow, Kafka, and Terraform.
• Optimize large-scale data pipelines for performance, reliability, and efficiency.
• Develop data processing workflows using Databricks, Delta Lake, and Spark.
• Maintain and improve a Data Lakehouse using Unity Catalog.
• Build automation, frameworks, and tools to streamline data engineering workflows.
• Collaborate across teams to ensure smooth data flow and integration.
• Apply best practices for observability, data governance, security, and regulatory compliance.

What you'll need

• Minimum 7 years of experience as a Data Engineer or similar role.
• Hands-on experience with Databricks, Delta Lake, Spark, and Scala.
• Ability to design, build, and operate Data Lakes or Data Warehouses.
• Proficiency with data orchestration tools such as Airflow, Dagster, or Prefect.
• Familiarity with change data capture tools including Canal, Debezium, or Maxwell.
• Strong command of at least one primary language (Scala or Python) and SQL.
• Experience with data catalog/metadata management (Unity Catalog, Lakeformation) and IaC with Terraform.