South Asian Software Engineers: Coding Tasks for AI Evaluation

Overview

Participate in a paid, AI-moderated study evaluating coding evaluation harnesses used to test AI agents. Review programming tasks and environments and provide technical feedback focused on realistic software-engineering requirements.

What you'll do

• Review proposed coding tasks and programming environments for realism.
• Provide technical feedback on the structure and difficulty of evaluation harnesses.
• Walk through your approach to verifying complex code challenges.
• Suggest improvements to make the test suites more robust.

What you'll need

• Be a professional software engineer based in South Asia.
• Have experience building or reviewing realistic programming tasks.
• Have familiarity with coding evaluation harnesses or test automation.
• Be comfortable discussing technical workflows in an interview setting.

Details

• Location: India.
• The study is paid at $35 per hour.
• Participation involves an AI-moderated session.