Bigdata Spark Developer

Overview

Intermediate Big Data Spark Developer role responsible for designing, developing, enhancing and supporting large-scale data processing solutions within the Applications Development function at Citi.

What you'll do

• Design, develop, test and maintain data processing applications using Apache Spark.
• Build scalable ETL and data transformation pipelines for large datasets.
• Develop and support applications using Java or Python in a distributed data environment.
• Analyze requirements and collaborate with business analysts, QA teams and stakeholders to deliver solutions.
• Troubleshoot production issues and optimize application performance.
• Participate in code reviews and follow engineering best practices.
• Contribute to application modernization, migration and automation initiatives.
• Leverage AI-assisted development tools to improve productivity and quality.
• Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas, to identify and define necessary system enhancements, including using script tools and analyzing/interpreting code.
• Consult with users, clients and other technology groups on issues, and recommend programming solutions; install and support customer exposure systems.
• Apply fundamental knowledge of programming languages for design specifications.
• Analyze applications to identify vulnerabilities and security issues, and conduct testing and debugging.
• Serve as advisor or coach to new or lower level analysts.
• Identify problems, analyze information and make evaluative judgements to recommend and implement solutions.
• Act as SME to senior stakeholders and/or other team members.

What you'll need

• 5-7 years of software development experience.
• Strong hands-on experience with Apache Spark.
• Proficiency in either Java or Python.
• Candidates with primary expertise in Python should have a good understanding of Java fundamentals (OOP concepts, collections, exception handling, multithreading basics).
• Experience working with SQL and relational databases.
• Understanding of distributed computing concepts and big data processing.
• Strong analytical, problem-solving and debugging skills.
• Demonstrated ability and willingness to adopt AI-assisted development tools (e.g., GitHub Copilot, Devin, Databricks Assistant, or similar) to improve productivity, code quality and delivery efficiency.
• Ability to leverage AI-generated recommendations for coding, testing, debugging, documentation and analysis while maintaining accountability for final deliverables.
• Strong engineering judgment to critically review, validate and optimize AI-generated code and solutions.
• Capable of identifying potential functional, performance, security and maintainability issues in AI-assisted implementations and making appropriate corrections.
• Passion for continuous learning and staying current with emerging AI technologies and software engineering best practices.

Nice to have

• Experience with Databricks (highly desirable).
• Exposure to cloud platforms and modern data engineering practices.
• Experience with CI/CD pipelines, Git and Agile development methodologies.
• Knowledge of regulatory reporting or financial services domain.
• Experience with Spark performance tuning.
• Familiarity with data lake architectures.
• Experience leveraging AI tools such as Copilot or similar developer productivity solutions.

Details

• Location: Pune, Maharashtra, India.
• Time type: Full time.
• Operates with a limited level of direct supervision and exercises independence of judgement and autonomy.
• Other job-related duties may be assigned as required.