Data Platform Engineer

Overview

Build and operate the shared data platform that stores, moves, and serves data across Alpaca, processing hundreds of millions of events each day, as a fully distributed, remote role.

What you'll do

• Design, develop, and evolve the core data platform, including distributed query engines, orchestration, lakehouse storage, metadata, and cataloging.
• Own lakehouse infrastructure as code, building reliable and repeatable deployment workflows with Terraform and Ansible on Kubernetes.
• Build and operate scalable, low-latency streaming and CDC pipelines alongside dependable batch ingestion paths into Apache Iceberg.
• Develop and scale serving and BI infrastructure, giving downstream teams and agents performant, governed, self-service access to lakehouse data.
• Take end-to-end ownership of platform reliability through observability, actionable alerting, on-call practices, incident response, maintenance procedures, runbooks, and service-level objectives.
• Partner with DevOps, Analytics Engineering, and stakeholders across Alpaca to close infrastructure gaps, shape technical requirements, and deliver reusable platform capabilities.

What you'll need

• 4+ years of experience in software engineering, data infrastructure, or platform engineering, with demonstrated ownership of complex production systems and technical projects.
• Strong understanding of distributed systems and production experience with query engines such as Trino or Presto.
• Strong experience operating Kubernetes-based data infrastructure using infrastructure-as-code and deployment tools such as Terraform, Helm, Ansible, or Argo CD.
• Experience designing and operating compute platforms across multiple cloud regions, balancing workload locality, peak and bursty demand, resource isolation, workload prioritization, and cost-performance trade-offs.
• Hands-on experience designing and operating cloud lakehouse architectures on GCP, AWS, or Azure, including object storage, metastores, and open table formats, particularly Apache Iceberg.
• Experience operating large-scale streaming and CDC systems using technologies such as Kafka, Redpanda, and Debezium.
• Strong Python and SQL skills, with experience building reliable data pipelines and platform tooling using orchestration and ingestion frameworks such as Airflow and Airbyte.
• Strong operational judgment and the ability to make sound technical decisions, communicate trade-offs, and work effectively through ambiguity.

Nice to have

• Experience with semantic or metrics layers such as Cube, dbt, or Looker.
• Experience deploying and operating infrastructure for AI agents.
• Experience building or integrating context layers that connect structured and unstructured organizational data.
• Experience with data governance and access-control frameworks such as Apache Ranger.

Details

• Remote, with the team 100% distributed (location line: Remote - Americas).