Pyspark Data Engineer

Overview

Data Engineer role focused on designing, developing, and maintaining scalable data pipelines using PySpark and Cloudera Data Platform (CDP).

What you'll do

• Design and develop scalable data pipelines using PySpark and CDP.
• Build, optimize, and maintain data ingestion and transformation processes.
• Ensure data quality, integrity, and availability across platforms.
• Work with large-scale datasets and big data ecosystems.
• Collaborate with cross-functional teams to deliver data-driven solutions.
• Monitor and troubleshoot data workflows and performance issues.

What you'll need

• Strong experience in PySpark and Cloudera Data Platform (CDP).
• Hands-on experience with Hadoop, Hive, HDFS, Spark, and ETL processes.
• Good understanding of data warehousing and big data architecture.
• Experience with cloud and distributed data processing environments.
• Strong problem-solving and analytical skills.

Details

• Location: Chennai.