Senior Engineer – Batch ETL, Big Data & Workflow Orchestration

Overview

Senior engineer role on LPL's Global Capability Center team in Hyderabad, building batch ETL and workflow orchestration for regulatory, supervision and analytics platforms within the Account Lifecycle Management domain.

What you'll do

• Design, develop, and optimize enterprise-scale batch ETL pipelines supporting LPL's regulatory, supervision, and analytics platforms.
• Lead ETL development using Informatica and PySpark for high-volume and complex data transformations.
• Build and maintain Airflow DAGs to orchestrate batch workflows, dependencies, retries, SLAs, and monitoring.
• Design scalable solutions for large data processing on distributed and cloud-based data platforms.
• Drive performance tuning, resiliency, data quality, and operational stability of batch processing jobs.
• Partner with architecture, upstream/downstream system teams, and business stakeholders to translate requirements into robust, compliant data solutions.
• Enforce LPL standards for security, data governance, auditability, and operational readiness.
• Perform code reviews, mentor Engineer I/II resources, and raise overall engineering maturity of the team.

What you'll need

• Strong hands-on experience with batch ETL architectures in regulated environments.
• Advanced expertise in Informatica.
• Strong proficiency in PySpark and distributed data processing concepts.
• Hands-on experience with Apache Airflow (authoring DAGs, scheduling, dependency management, monitoring).
• Experience processing large datasets with a focus on performance and reliability.
• Strong SQL and data modeling fundamentals.
• Ability to own complex problem areas end-to-end and operate with minimal supervision.

Details

• Location: Hyderabad.