Data Engineer

Overview

Join FOX's Data and Commercial Technology Team as a Data Engineer in the Engineering Centre of Excellence, building robust, scalable, efficient, and high-quality data engineering solutions for the Enterprise Data Platform.

What you'll do

• Collaborate with and across Agile teams to design and develop data engineering solutions by rapidly delivering value to customers.
• Build distributed, low latency, reliable data pipelines ensuring high availability and timely delivery of data.
• Design and develop highly optimized data engineering solutions for Big Data workloads to handle increasing data volume and complexity.
• Build high-performing real-time data ingestion solutions for streaming workloads.
• Adhere to best practices and agreed-upon design patterns across Data Engineering solutions.
• Design, code, and tune solutions for performance.
• Implement processes and systems to monitor data quality, ensuring production data is accurate and available.
• Create design documentation, including Data Flow Diagrams, Technical Design Specs, and Source to Target Mapping documents.
• Create and execute unit and integration tests.
• Perform data analysis to troubleshoot data-related issues and assist in resolving data issues.
• Automate data engineering pipelines and data validations, including audit and balance controls, without manual intervention.
• Implement data security and privacy through access controls, key management, and encryption techniques.
• Learn new technologies, stay on top of technology trends, experiment with new tools and technologies, and educate other team members.
• Collaborate with analytics and business teams to improve data models that feed business intelligence tools, increasing data accessibility and supporting data-driven decision-making.
• Communicate clearly and effectively with technical and non-technical leadership.

What you'll need

• Bachelor’s degree in Computer Science, Computer Engineering, or a relevant field.
• Data Engineer: 4+ years of experience designing and building Data Engineering solutions.
• Senior Data Engineer: 7+ years of experience architecting, designing, and building Data Engineering solutions and Data Platforms.
• Experience building Data Warehouses/Data Platforms on Redshift or Snowflake.
• Experience working with data acquisition and transformation tools such as Fivetran and DBT.
• Experience building highly optimized and efficient data engineering pipelines using Python and PySpark.
• Experience working with distributed data processing frameworks such as Apache Hadoop, Apache Spark, or Flink.
• Experience working with real-time data stream processing using Apache Kafka, Kinesis, or Flink.
• Experience working with AWS Services including S3, EC2, EMR, Lambda, RDS, DynamoDB, Redshift, and Glue Catalog.
• Expertise in Advanced SQL programming and SQL Performance Tuning.
• Experience with version control tools such as GitHub or Bitbucket.
• Expert-level understanding of dimensional modeling techniques.
• Excellent communication, adaptability, and collaboration skills.
• Excellent analytical skills and strong attention to detail, with emphasis on accuracy, consistency, and quality.
• Strong logical and problem-solving skills with critical thinking.

Nice to have

• Experience designing and building applications using container and serverless technologies.
• Experience working with fully automated workflow scheduling and orchestration services such as Apache Airflow.
• Experience working with semi-structured and unstructured data and NoSQL databases.
• Experience with CI/CD using GitHub Actions or Jenkins.
• Experience designing and building APIs.

Details

• Location: Bengaluru, Karnataka, India.