Data Engineer – Apache Spark | Kafka | Flink | Trino | Iceberg | Big Data | Streaming | Data Platform 4–8 Years

Overview

Join the Webex Data Analytics Platform engineering organization as a hands-on Data Engineer focused on developing the big data platform and working with the product team to develop data and analytics pipelines.

What you'll do

• Work on and enhance big data-related open-source technologies.
• Build and operate Kafka-based streaming applications, including ingestion, filtering, enrichment, and replication use cases.
• Develop data processing jobs using Apache Spark and/or Apache Flink, following established patterns and platform guidelines.
• Work with Apache Iceberg data lake technologies to manage large analytical datasets.
• Tune jobs for performance, scalability, and cost efficiency.
• Write clean, testable code.
• Participate actively in code reviews.

What you'll need

• At least 4-8 years of experience in data engineering and software development.
• Ability to work with the open-source ecosystem.
• Ability to write high-quality code in Java/Scala, Python, or equivalent languages.
• Practical experience with Apache Spark and Apache Flink.
• Practical experience with query engines like Trino/Presto.
• Experience working with Kafka and scaling for high-volume workloads.
• Strong SQL skills.
• Understanding of data modeling for analytical workloads.

Nice to have

• Hands-on experience with real-time analytics stores like Apache Pinot/ClickHouse for low-latency analytical queries.
• Familiarity using AI tools for debugging and development, including CoPilot, Cursor, and Codex.
• Familiarity with data lake table formats such as Apache Iceberg.
• Familiarity with data governance tools like DataHub.
• Experience working with tools like DolphinScheduler or Airflow.
• Experience working with lakehouse management systems like Apache Amoro.

Details

• Location: Bangalore, India.