Staff Big Data Engineer

Overview

Staff Big Data Engineer to define and evolve a data platform and pipeline architecture for Qualys' Enterprise TruRisk Platform.

What you'll do

• Define and drive the technical vision and long-term strategy for the data platform and pipeline ecosystem.
• Design, build, and maintain scalable, high-availability data processing systems using Apache Spark and Apache Kafka.
• Lead architecture and design decisions across multiple engineering teams.
• Implement event-driven, streaming, and large-scale data processing architectures for real-time and batch workloads.
• Troubleshoot and optimize complex performance bottlenecks across data pipelines and distributed systems.
• Partner with Product Management and other teams to evaluate technical solutions and architecture trade-offs.

What you'll need

• Minimum 12 years of experience in software engineering/data engineering/distributed systems/big data platform development.
• Minimum 6 years of hands-on experience with Apache Spark-based data processing solutions.
• Minimum 6 years building/operating large-scale data pipelines processing billions of events or transactions daily.
• Minimum 4 years administering and operating Apache Kafka in production.
• Minimum 4 years designing and implementing event-driven or streaming data architectures.
• Bachelor’s degree in Computer Science, Computer Engineering, Information Technology, or a related technical discipline.

Nice to have

• Experience working with high-volume, multi-tenant enterprise platform environments.
• Proficiency in Scala, Java, or Python for data processing applications.
• Proven experience mentoring junior or mid-level engineering team members.

Details

• Provide technical leadership across multiple engineering teams.