Mindera logo

Senior Data Engineer

Mindera
NewPosted yesterday

LOCATION

Porto · Remote

EXPERIENCE

Not specified

SKILLS REQUIRED

PythonSQLData PipelinesData WarehousingSnowflakeDBTApache AirflowPySparkDatabricksApache Kafka

Job description

Overview

Build and run the pipelines and platforms that move customers' data from source systems to the warehouse, on production systems with real consequences.

What you'll do

  • Build and run data pipelines and platforms that move customers' data from source systems to the warehouse.

What you'll need

  • Strong hands-on experience with Python and SQL, using both in production environments.
  • Experience building batch and streaming pipelines from databases, event streams, and APIs.
  • Solid production experience in AWS.
  • Experience with data orchestration tools like Airflow, Dagster, Azure Data Factory, or similar engines.
  • Good knowledge of CI/CD and Infrastructure as Code, using tools such as Git, GitHub Actions, Terraform and Docker.
  • Experience with automated testing, monitoring, alerts and troubleshooting to ensure pipelines run reliably and recover from failures.
  • Ability to manage data contracts with upstream teams, managing schema drift, and handling data access and lineage.
  • Regular use of AI developer tools, with the ability to validate, review and ensure the quality of AI-generated code.
  • Experience with Snowflake and dbt, including data transformation and modelling (Dimensional or Data Vault).
  • Hands-on experience with distributed data processing and streaming technologies, such as Spark, PySpark, Databricks, Kafka, Kinesis or Google Pub/Sub.
  • Fluent English for technical and non-technical communication.

Nice to have

  • Experience with GCP and Azure.
  • Awareness of GDPR.
  • NoSQL databases (MongoDB) experience.
  • Marketing/CRM APIs experience.

Details

  • Location: Porto, Portugal (hubs in Porto, Aveiro, or Coimbra, or remote as long as based in Portugal).
  • Expected travel: 0%-15%, national and international, depending on project/client and organizational needs.

Read the full description and apply on the company’s own careers page.

Stay safe

Hiring on Abekus is free for applicants

We never charge a fee, and employers are prohibited from doing so. If a recruiter asks for payment, please report them right away.

Senior Data Engineer