Terac
More roles at TeracSoftware Engineers: Paid Interview on AI Evaluation Tasks
Remote role where the employee must remain based in a particular country.
United States only
From the listing details: Source listing states candidate location: "United States, Remote"
Employer listed it 5 weeks ago · Added 5 weeks ago
Been open since 5 weeks ago, still being checked, but it has been live a while.
Salary
Not stated
Location
United States only
Timezone
US East
Contract
Contract
Experience
Mid
Category
Software
The employer did not publish a salary, and we have too few comparable roles to estimate one.
What the advert says
Quoted word for word from the employer's advert. Where it says nothing, we say so.
- Working hours
- Not mentioned in the advert
- Where you can work from
- Location eligibility not confirmed
- Contract
- Not mentioned in the advert
- Experience asked for
- Not mentioned in the advert
- Pay
Compensation
Remote flexibility
Work from home
This is a remote role, but the employee must be based in United States. It is work from home rather than work from anywhere.
What the employer says
- Source listing states candidate location: "United States, Remote"
What Nomaders makes of it
- Residency required in United States
- Payroll and tax are likely handled in that country only
The quotes above are the employer's own words; the reading is ours. Always check the original listing and employment terms before working from another country.
Before you apply
Questions worth asking: Nomaders interpretation
- The advert does not say where you can be based. Ask which countries they can legally employ or contract you from.
- The advert does not say whether this is employment or a contractor arrangement. Ask, as it affects tax and benefits.
Produced automatically from this advert's text; not reviewed by a person. The employer's own page is the final word.
About the role
What We're Researching
We're running a paid study on the quality and realism of coding environments designed to test AI agents. We are developing a comprehensive suite of programming tasks and evaluation harnesses to measure AI performance accurately. Your feedback directly shapes how these agents are benchmarked against real-world engineering standards.
How It Works
During this remote session, you will review several programming tasks and their corresponding evaluation harnesses. You will assess the technical accuracy, complexity, and realism of each coding challenge. We will ask you to walk through the logic of the evaluation environments and identify any potential flaws. Finally, you will provide feedback on how these environments compare to standard industry practices.
Who This Is For
We are looking for software engineers who have hands-on experience building, reviewing, or testing realistic programming tasks. We welcome full-stack developers, backend engineers, test automation engineers, and systems architects who are familiar with evaluation harnesses. Ideal candidates understand what makes a coding challenge robust, verifiable, and technically sound.
What You'll Do
Review and assess the quality of realistic programming tasks
Evaluate coding environments and technical harnesses for AI testing
Walk us through potential flaws in the provided code structures
Provide feedback on the difficulty and realism of the challenges
Who Should Apply
Professional experience as a software engineer or developer
Familiarity with building or verifying programming tasks
Experience with code review and evaluation harnesses
Comfortable discussing technical architecture and testing methodologies
Compensation
$75 per hour
Ready to participate?
Start your paid interview now
About Terac
Terac is building the world's largest pool of vetted human experts for AI. Researchers, AI labs, and product teams use Terac to recruit, screen, and pay study participants across industries, languages, and skill sets.
Learn more at terac.com or on YouTube at @jointerac .
Requirements
The employer hasn't listed requirements separately, they're described in the role summary above and on the original listing.
Benefits
No benefits package published with this listing. Ask about it at first interview.
How to apply
- 1Check the flexibility label above, work from home, matches where you plan to live and work.
- 2Tailor your CV to the role at Terac, mentioning your remote working experience and working hours (US East).
- 3Apply directly on the employer's careers page using the button below. Nomaders never handles your application.
Found 1mo ago. Last checked today. Always confirm the details on the original posting, salary and location can change after publication.
Listing sourced from Company boards.
Pay not stated · You'll be taken to the employer's careers page.