Staff Data Platform Engineer

Overview

Lead the data platform that makes CertifyOS the source of truth for provider data, setting architecture, leading a team of engineers, and owning the reliability, freshness, and cost of that data.

What you'll do

• Lead a team of engineers to deliver data platform initiatives end to end
• Own data architecture across OLTP, OLAP, and NoSQL stores (Aurora PostgreSQL, Redshift, MongoDB)
• Design and operate distributed real-time and batch pipelines in Python and PySpark (Kafka, Kinesis, SNS/SQS)
• Lead entity resolution and MDM architecture for provider data
• Build and own data products with clear contracts, SLAs, and consumers
• Be accountable for data reliability, freshness, quality, and cost, with the observability to prove it
• Drive performance and scale work: profile, benchmark, and re-architect as volume grows
• Build the data foundations for AI/ML workloads
• Mentor engineers, raise the bar on design and code review, and partner with product, analytics, and ML
• Join the on-call rotation and turn incidents into systemic fixes

What you'll need

• 10+ years building production data systems, including Staff or Tech Lead scope
• Led a team of engineers to deliver complex data systems
• Expert Python and production PySpark
• Deep distributed systems knowledge: consistency, partitioning, idempotency, backpressure, failure handling
• Significant entity resolution and MDM experience: matching, deduplication, survivorship, golden records at scale
• Built and owned data products used by other teams
• Shipped measurable performance and scale improvements
• Expertise across OLTP (Aurora, PostgreSQL, MySQL), OLAP (Redshift, Athena, Snowflake, ClickHouse), and NoSQL (MongoDB, DynamoDB, Cassandra)
• Stream processing and event-driven architecture (Kafka, Kinesis, SNS/SQS)
• Data observability in practice: quality checks, lineage, monitoring, alerting
• Built data platforms that serve ML workloads
• Influences technical direction across teams and communicates clearly with non-technical stakeholders
• Experience handling sensitive data (PII/PHI)
• Fluent with AI-assisted development (Cursor, Claude Code, MCP)

Nice to have

• Change Data Capture (CDC) patterns
• Data mesh or data-platform-as-a-product approaches
• Schema registries or data catalogs
• Orchestration tools (AWS Glue, EMR, Managed Flink, Airflow/MWAA, dbt)
• Healthcare or other regulated environments

Details

• Location: Bengaluru, India