Senior Security Data Engineer

Overview

Senior Security Data Engineer focused on building and managing high-throughput streaming data pipelines for security log ingestion, normalization, routing, and preparation for analytics/AI-ML detection models.

What you'll do

• Maintain streaming pipelines to ingest logs from firewalls, cloud, and endpoints.
• Build parsers to normalize raw logs into standard schemas (e.g., OCSF or ECS).
• Implement routing logic to send high-value data to a SIEM and bulk data to object storage.
• Clean and structure data to support AI/ML detection models and advanced analytics.

What you'll need

• Have proficiency in Python for ETL and SQL for complex querying.
• Experience with message queues (e.g., Kafka, Pub/Sub) and stream processing concepts.
• Master regex and log parsing strategies for standard formats (Syslog, CEF, JSON).
• Understand data lake principles and formats (Parquet/Avro) versus data warehouses.
• Experience with vector databases for storing embeddings.
• Knowledge of log observability/routing tools (middleware that routes logs).
• Familiarity with big data frameworks such as Spark and Flink.

Details

• Base salary range is 35,000–45,000.