MLOps Engineer (Focusing AWS platform) - RDT Pharma Medicines

Overview

Design, build and maintain scalable AWS-based data pipelines and infrastructure for commercial pharma data, working independently to own projects end to end.

What you'll do

• Lead the design, build and maintenance of scalable data pipelines.
• Independently execute data engineering projects from start to finish.
• Build ETL/ELT processes and optimize data storage solutions such as data warehouses, data lakes and lakehouses.
• Solve complex data ingestion and processing problems and optimize data flows for system performance.
• Ensure data quality and reliability and monitor data systems.
• Engage with business units, business analysts and data scientists to understand and meet their data needs.
• Communicate effectively with both technical and non-technical stakeholders.
• Implement strategies that significantly improve the organization's data infrastructure.
• Integrate multiple diverse data sources and ensure efficient, reliable data flow across platforms.

What you'll need

• 6-10 years in data engineering with proven independent delivery of large-scale pipelines.
• Demonstrated experience handling big-sized data engineering projects and managing complex, large-scale data systems.
• Proven track record of independently building ETL/ELT processes and optimizing data storage solutions such as data warehouses and data lakes.
• Experience ensuring data quality, reliability, and monitoring data systems.
• Proficiency in programming languages such as SQL, Python, PySpark.
• Proficiency in AWS cloud-based data engineering: S3, Lambda, AWS Glue, Athena, Airflow, DAGs, RedShift, RDS, AWS DataSync, AWS Step Functions, QuickSight, dbt.
• Strong, expert knowledge of cloud platforms and Big Data tools, specifically Spark.
• Strong data modeling (dimensional, data vault, lakehouse), partitioning and file formats.
• Data quality, monitoring and incident troubleshooting experience.
• Experience building data products, designing a semantic layer, and handling unstructured data.
• Terraform or CloudFormation, Git-based CI/CD, containerization.
• Advanced capability to solve complex data ingestion and processing problems, and to optimize data flows for enhanced system performance.

Nice to have

• Exceptional communication and collaboration skills, with the ability to engage both technical audiences (such as data scientists) and non-technical business stakeholders.
• HCP/HCO master data (Veeva Network, OneKey), patient-level longitudinal data, de-identification.
• Commercial use cases: sales performance, market access, patient journey, next best action.
• AWS certification (Data Engineer Associate, Solutions Architect, Big Data Specialty).

Details

• Location: Hyderabad.