Senior Data Engineer

Overview

Design, build and optimize modern cloud-based data platforms that power analytics, AI and data products, while helping shape business requirements.

What you'll do

• Design and implement scalable data platforms and pipelines, primarily on AWS and Databricks, and on GCP or Azure/Fabric where projects require.
• Develop reliable batch, streaming, and near-real-time pipelines using Spark and Delta Lake.
• Build ingestion, transformation, and curation workflows for structured and unstructured data.
• Implement lakehouse patterns and medallion layering (bronze, silver, gold), ensuring reusable, scalable systems aligned with enterprise needs.
• Contribute to cloud-to-cloud migrations, including re-platforming pipelines and data models.
• Deliver model-ready, production-grade datasets for analytics, ML, causal modeling, and optimization.
• Build data pipelines for GenAI use cases (LLMs, RAG, vector-based data flows) and agent-based architectures.
• Leverage Databricks MLflow and Unity Catalog, or equivalent tooling on other platforms.
• Design logical and physical data models for analytical and operational use cases.
• Orchestrate workflows with Databricks Workflows, Airflow, dbt, or equivalents (e.g., Azure Data Factory, Cloud Composer), with a focus on automation, reliability, and maintainability.
• Apply event-driven and streaming architecture patterns.
• Ensure data governance, lineage, quality, and access control (RBAC/ABAC) using Unity Catalog, AWS Lake Formation, or equivalents (Microsoft Purview, GCP Dataplex).
• Establish observability: data freshness, pipeline reliability, and SLA adherence.
• Enable serving layers (APIs, feature inputs, analytical endpoints) for downstream ML and AI platforms.
• Continuously optimize pipelines and infrastructure for performance, scalability, and cost.
• Ask the right questions to understand not just what stakeholders want, but why.
• Surface gaps and better alternatives rather than executing a brief as written.
• Work closely with data scientists, ML engineers, analysts, and business stakeholders.
• Support adoption of data products and contribute to best practices.

What you'll need

• Strong hands-on experience with Apache Spark and Delta Lake.
• Strong Python and SQL.
• Proven experience building batch and streaming pipelines and production-grade data platforms.
• Solid understanding of data modeling, data quality, and governance.
• Strong, demonstrable hands-on experience with the Databricks Lakehouse Platform, ideally on AWS (e.g., S3, Glue, EMR, Redshift, Kinesis, Lambda).
• Deep expertise in at least one major cloud (AWS, GCP, or Azure/Fabric), plus demonstrable openness and ability to work across clouds.
• Candidates with a strong Azure/Fabric background (Data Factory, Synapse, Fabric Lakehouse, Azure Databricks) are encouraged to apply, provided they are keen to work in other clouds.
• Experience with lakehouse architectures and distributed systems.
• Strong understanding of scalability, reliability, and performance.
• Naturally curious, with strong problem-solving skills and a collaborative approach in cross-functional teams.
• Comfortable challenging business requirements rather than taking them at face value.
• Adaptable and eager to work across platforms.

Nice to have

• GCP experience (e.g., BigQuery, Dataflow, Pub/Sub, Cloud Storage, Dataproc), especially in migration contexts, is highly valued.
• Agile or consulting experience is beneficial.
• Hands-on experience across more than one cloud, or cloud migration experience (e.g., Azure to GCP).
• Snowflake experience.
• GenAI and AI data systems (RAG pipelines, vector databases, LLM data preparation).
• CI/CD for data pipelines and infrastructure-as-code (Terraform, CloudFormation).
• Streaming technologies (e.g., Kafka), Spark optimization, or ML/experimentation platforms.
• Experience building data products or large-scale analytics platforms.

Details

• Location: Poland.
• Remote and hybrid work options (country-dependent).
• Staffed on projects that fit your strongest platform, with opportunities to expand.