Data Platform Engineer, Vice President II - State Street Investment Management

Overview

Hands-on architect and engineer role (80% individual contributor, 20% people management) building a multi-cloud, open-architecture data platform that delivers trusted data products to business functions at State Street Investment Management, based in Bangalore.

What you'll do

• Validate target data platform architectures and design principles through hands-on engineering, technical spikes, and POCs for a unified, cloud-native Lakehouse.
• Partner with architects to translate reference architectures and data strategies into working prototypes, measurable validation criteria, and production-ready implementation patterns.
• Optimize scalable batch and streaming data pipelines using Python/Scala, SQL, Spark, and modern orchestration frameworks.
• Engineer high-performance Iceberg-based data models and tables, including partitioning, compaction, schema evolution, metadata management, and multi-engine interoperability.
• Establish data engineering standards for data quality, lineage, observability, reliability, versioning, and backward-compatible data product contracts.
• Optimize platform, pipeline, and query performance through workload testing, partitioning and clustering, caching, concurrency management, and cost-aware lifecycle practices.
• Lead capabilities from POC to production by defining non-functional requirements, resolving engineering risks, integrating with platform services, and enabling operational support and SLA readiness.
• Engineer AI-ready data products with trusted, well-governed, semantically enriched data, incorporating context engineering, ontologies, embeddings, vector indexing, and RAG patterns.
• Build and productionize data services for Agentic and AI workloads, including MCP-enabled access, interoperable APIs, LLM gateways, multi-model routing, caching, observability, security, and cost optimization.
• Mentor and provide technical leadership to the L2/L3 support teams for the legacy platform to ensure SLA adherence, escalation discipline, and timely closure of production issues impacting business, clients, or regulators.

What you'll need

• Master's degree in computer science or a related technical discipline.
• 15+ years of technology experience, preferably in financial services.
• Expert-level hands-on programming in Python and SQL; proven ability to design, debug, test, and optimize distributed data-processing solutions.
• Java or Scala (listed as a plus in the source text, noted here for context).
• Deep hands-on expertise with Apache Spark for batch and streaming workloads, including performance tuning, troubleshooting, reliability engineering, and cost optimization.
• Proven experience building and operating enterprise Lakehouse and data platforms using Snowflake and Databricks, supporting high-concurrency analytical, near-real-time, and AI-ready workloads.
• Strong cloud engineering experience across AWS, GCP, and Azure, including cloud-native data services, security, networking, infrastructure automation, portability, and cost management.
• Working expertise with open data architectures and metadata ecosystems, including Apache Iceberg, schema evolution, table optimization, catalog federation, zero-copy sharing, data virtualization, and multi-engine access.
• Extensive experience productionizing data capabilities through DevOps and CI/CD, automated testing, observability, data quality, incident management, and operational support for mission-critical platforms.

Nice to have

• Experience with both structured and unstructured data management tools and associated patterns.
• Familiarity with modern governance frameworks (Unity Catalog, Collibra, Alation).
• Expertise in AI-assisted coding using GitHub Copilot or Claude Code Assistant.

Details

• Location: Bangalore, India.
• Role split: approximately 80% hands-on engineering and POCs, 20% managing and mentoring employees responsible for production maintenance and support of the legacy data platform.