Architect - Big Data Application

Overview

Architect (Big Data Applications) focused on designing and leading distributed data processing solutions using Apache Spark and modern data platform technologies.

What you'll do

• Design scalable, resilient Spark-based data processing solutions.
• Create architecture designs, reference patterns, and technical specifications.
• Provide technical leadership across batch and streaming Spark workloads.
• Drive performance improvements for Spark via optimization and tuning strategies.
• Conduct architecture and design reviews across the software development lifecycle.
• Establish standards for observability, monitoring, reliability, and operational excellence.

What you'll need

• 10+ years of experience in software engineering, data engineering, or distributed systems development.
• Hands-on expertise in Apache Spark (Spark SQL, Structured Streaming, DataFrames, Dataset APIs).
• Experience designing and building large-scale distributed data processing systems.
• Strong knowledge of Spark optimization techniques (partitioning, shuffle, join, memory, resource utilization, tuning).
• Proficiency in Scala, Java, or Python.
• Experience deploying Spark solutions on AWS and/or GCP platforms.
• Understanding of cloud-native architecture and distributed systems principles.

Details

• Hybrid role with regular in-person work at an assigned TU office location for at least two days per week.
• Location: Hyderabad.