Overview
Big Data Specialist role managing and optimizing data integration and ETL processes that move data across systems to support decision-making and operational efficiency.
What you'll do
- Develop, design and implement ETL processes using Informatica PowerCenter, PySpark, Hive, Hadoop, HDFS, Impala, Spark, Spark SQL and Kafka to ensure efficient data extraction, transformation and loading.
- Collaborate with cross-functional teams to gather data requirements and translate them into technical specifications.
- Monitor and maintain existing ETL workflows, troubleshoot issues and perform necessary optimization.
- Work on data quality and leasing initiatives to ensure accurate and reliable data.
- Participate in the design and development of data warehouses, data marts and other data storage solutions.
- Stay updated with industry best practices and emerging trends in data integration and ETL.
What you'll need
- A bachelor's degree in Computer Science, Information Technology or a related field.
- Proven experience of 3-5 years in designing, developing and implementing automation processes using PySpark and Hive.
- Strong SQL skills and proficiency in working with relational databases (e.g. Oracle, SQL Server, MySQL).
- Solid understanding of data warehousing concepts, data modeling and ETL best practices.
- Experience in performance tuning, troubleshooting and debugging ETL processes.
- Excellent problem-solving skills and ability to work effectively in a collaborative team environment.
- Strong communication skills to interact with technical and non-technical stakeholders.
Nice to have
- Previous exposure to cloud-based data integration using Informatica Cloud or other cloud ETL tools.
- Experience with Big Data technologies (Hadoop, Spark) and SQL & PL/SQL databases.
- Knowledge of data privacy regulation and best practices for handling sensitive data.
- History of leading successful ETL projects from design to implementation.
- Strong analytical skills for data profiling, data mapping and data assessment.
Details
- Location: Mumbai, India.
- Hybrid work setting.
- Full-time employment.
Read the full description and apply on the company’s own careers page.