Senior Software Engineer

Overview

Senior Software Engineer role building high-performance distributed data systems and infrastructure for industrial data at Cognite's India R&D Center.

What you'll do

• Build high-performance data pipelines using Spark, Flink, and Kafka to process terabyte-scale industrial datasets.
• Develop low-latency APIs and services supporting thousands of concurrent users with sub-second response times.
• Optimize time-series, sensor, and operational data storage and retrieval for massive scale.
• Engineer distributed processing solutions, including real-time streaming that handles millions of events per second.
• Design and evolve cost-efficient data lake architectures (S3/GCS) using modern formats like Parquet/ORC.
• Optimize OLAP query performance across systems such as ClickHouse, Pinot, or Druid.
• Build fault-tolerant, highly-available systems with circuit breakers, retries, and distributed tracing.
• Drive technical innovation through research, prototyping, and contributions to open-source.
• Collaborate with product and architecture teams to translate complex industrial data problems into scalable, production-grade solutions.

What you'll need

• 6–12 years of experience in distributed data systems, backend engineering, or platform engineering.
• Deep expertise in Spark internals, including Catalyst optimizer, partitioning, and performance tuning.
• Strong hands-on experience with Flink/Kafka for both streaming and batch workloads.
• Proven ability to scale systems to 10K+ QPS with strong foundations in JVM performance, GC tuning, and memory management.
• Experience building high-throughput REST/gRPC services and advanced caching with Redis/in-memory data structures.
• Strong understanding of monitoring and observability (Prometheus, Grafana, OpenTelemetry).
• Solid architectural thinking with an ability to design distributed, secure, and resilient platforms.
• Collaborative mindset with the ability to work across engineering, product, and platform teams.
• Strong programming skills in Scala, Java, or Python.
• Experience with cloud platforms (AWS/GCP/Azure) and data lake/object storage.

Nice to have

• Prior experience with large-scale data, OLAP systems, or industrial/IoT data.
• Contributions to open-source or experience in industrial data/AI platforms.

Details

• Location: Bengaluru, India (Cognite's India R&D Center).