Databricks QA Engineer

Overview

Own the quality gate for enterprise Databricks data platforms, validating Lakehouse pipelines and signing off releases working directly with customers.

What you'll do

• Own test strategy and release sign-off on engagements, including documented test plans covering scope, risk, and coverage
• Build and maintain automated test suites in Python and PySpark (pytest), integrated into CI pipelines
• Validate data pipelines end to end: ingestion, Medallion layer transformations, Delta Lake tables, and ELT workflows
• Implement data quality rules as code, including Lakeflow expectations and reusable validation frameworks
• Run regression validation after code changes and maintain CI quality gates that block merge or promotion on failure
• Partner with engineers to identify automation candidates and raise the coverage bar across the practice

What you'll need

• 5+ years in QA automation or test engineering, including meaningful time testing data pipelines rather than applications alone
• A data engineering background: you can read a pipeline, reason about a transformation, and write the SQL to verify it
• Expert-level SQL
• Experience with Medallion architecture, Delta Lake, and ELT-based data platforms
• Experience building automated suites that run in CI and gate deployment
• Demonstrable hands-on experience working on Databricks or similar platforms

Nice to have

• Healthcare or life sciences experience
• Existing Databricks certification(s)
• Hands-on experience with Unity Catalog, Lakeflow pipelines, or Databricks Asset Bundles
• Experience with Azure DevOps pipelines, or equivalent CI tooling (GitHub Actions, GitLab CI)
• Familiarity with contract-based data validation standards such as ODCS
• Experience using LLM agents or MCP-connected tooling to run validations, triage failures, or draft defect reports
• Consulting or client-facing background

Details

• Contract engagement, pay rate $70 – $80 per hour (1099 independent contractor)
• Open to contractors anywhere in the United States