Sr Data Engineer

Overview

Sr Data Engineer to design, develop, and optimize complex data pipelines, solutions, and metadata-driven architectures for enterprise data access and analytics with R&D functional knowledge.

What you'll do

• Design, develop, and maintain scalable ETL/ELT pipelines for structured, semi-structured, and unstructured processing.
• Implement real-time and batch data processing with unified, governed data fabric integration.
• Optimize distributed big data processing frameworks using Apache Spark, Hadoop, or similar technologies.
• Enable enterprise data discovery and governance using metadata management and data lineage tracking.
• Ensure data security, compliance, and RBAC across data environments.
• Build CI/CD pipelines for automated data pipeline deployments, version control, and monitoring.

What you'll need

• Hands-on data engineering experience with Databricks, PySpark, SparkSQL, Apache Spark, AWS, Python, and SQL.
• Workflow orchestration and performance tuning experience for big data processing.
• Strong understanding of AWS services.
• Experience with Data Fabric, Data Mesh, or similar enterprise-wide data architectures.
• Experience with Scaled Agile methodologies and Agile delivery practices.
• A Master’s degree with 3 to 4+ years, or a Bachelor’s degree with 5 to 8+ years of relevant experience.

Nice to have

• Deep expertise in Biotech & Pharma industries.
• Experience writing APIs to make data available to consumers.
• Experience with SQL/NoSQL databases and vector database use cases for large language models.

Details

• Prefer AWS Certified Data Engineer and Databricks certificate.
• Scaled Agile SAFe certification preferred.