South Asian Software Engineers: Coding Tasks for AI Evaluation
Company shown to members 🔒 · Remote · India only
Apply free: 3 jobs are on us✨ Tailor my resume for this job🎤 Practise the interview question, free
✓ Straight from the company's own careers page. No middlemen, no “we'll send you a check for equipment”. How we check jobs
✓ Sent 100 applications and heard nothing? Make this one count: tailor your resume to this ad in one tap, free.
✨ Not sure this is the one? Get your remote job matches in 30 seconds →
First recorded by Psst: 12 October 2026. Your first 3 jobs are free to apply to (the first right away, 2 more with your email). After that, the company and the apply link are for Psst members, from $4.99 for 7 days.
Practise the first interview question for this job
If the company replies, the first call usually starts with a question like this. Vague answers are the easiest to reject. Write yours and get an AI score in under a minute.
The 20-second read
- Experience
- Experience not stated
- Where you can live
- India only
- Job type
- Contract or freelance
Read from the ad by Psst's rules. Check the full ad below before applying.
Job details
- Where you can work
- India
- Employment type
- Contract
- Advertised pay
- $35/hr
- Experience
- Experience not stated
- Employer publication date
- 1 October 2026
- First recorded by Psst
- 12 October 2026
- Last checked at the source
- 11 November 2026
About the job
WHAT WE'RE RESEARCHING
We're running a paid study on the effectiveness of coding evaluation harnesses used to test AI agents. Our team is building a comprehensive suite of programming environments designed to measure agent performance accurately. This work ensures that AI coding assistants are evaluated against realistic, high-quality development scenarios.
HOW IT WORKS
You will walk through a series of proposed coding tasks and environments during an AI-moderated session. We will ask you to review the structure, difficulty, and realism of these programming challenges. You will provide technical feedback on the evaluation harnesses and suggest improvements to the test suites. The conversation will focus on ensuring these tasks accurately reflect real-world engineering requirements.
WHO THIS IS FOR
We are hiring experienced software engineers based in South Asia who have a strong background in building and reviewing complex codebases. We welcome backend developers, test automation engineers, and full-stack engineers who understand evaluation harnesses. Ideal candidates have hands-on experience verifying realistic programming tasks in professional environments.
WHAT YOU'LL DO
- Review proposed coding tasks and programming environments for realism
- Provide technical feedback on the structure and difficulty of evaluation harnesses
- Walk us through your approach to verifying complex code challenges
- Suggest improvements to make the test suites more robust
WHO SHOULD APPLY
- Professional software engineer based in South Asia
- Experience building or reviewing realistic programming tasks
- Familiarity with coding evaluation harnesses or test automation
- Comfortable discussing technical workflows in an interview setting
COMPENSATION
$35 per hour
READY TO PARTICIPATE?
Start your paid interview now [link for members]
ABOUT the company
The company is building the world's largest pool of vetted human experts for AI. Researchers, AI labs, and product teams use the company to recruit, screen, and pay study participants across industries, languages, and skill sets.
Learn more at the company [link for members] or on YouTube at @jointerac [link for members]
Apply free: 3 jobs are on us✨ Tailor my resume for this jobPrepare with the Apply kitMore remote software engineer jobs →
Posted recently with nothing unusual.
Signals, not proof. Check the company's page before you spend time on it.Similar remote jobs
- AI Evaluation EngineerIndia only
- TypeScript Coding Specialist - Freelance AI Trainer ProjectWork from anywhere
- Rust Coding Specialist - Freelance AI Trainer ProjectWork from anywhere
- Quantitative Finance Expert - AI Training & EvaluationUS, India, Pakistan only
- Investment Banking Expert - AI Training & EvaluationUS, India, Pakistan only
Explore more jobs
A resume that gets past the first 6 seconds
Recruiters skim. The Psst resume coach fixes what makes them skip you, line by line, and one tap tailors it to the exact job you're applying to.
- 1Every line checkedWeak starts, clichés and AI-sounding phrases flagged, with a 3-question fix.
- 2Tailored to the jobSee which skills and tools from the ad your resume is missing. Try it free on one of your matches.
- 3Ready for remoteTime zone, tools and proof you work on your own, the things remote teams look for.
- 4Looks the partOne page, 6 designs, 8 colours and a matching cover letter, free to build. Premium downloads the PDF and Word files.
Example with made-up details. The score is the builder's own check, not an ATS rating or a promise of an interview.