
Expert Talent Pool, Evals and Benchmarks (Associate)
Clera
WorldwideFully remoteContractEntry Level, JuniorInformation Technology
USD 30–60 an hourPosted 4 hours ago4h agoApply link checked 4 hours ago
Why we think you can apply
- Hiring model
- Open worldwide · the employer states no country restriction
- Work from
- Anywhere, the UAE included
Posted 4 hours ago · apply link checked 4 hours ago
About the role
Responsibilities
- Write questions and tasks that test specific model capabilities
- Develop rubrics defining strong, partial and weak answers
- Create reference answers and edge-case scenarios
- Grade model outputs against rubrics and explain each score
- Analyze incorrect answers, group error patterns and document findings
- Compare multiple models using the same prompts
- Validate benchmarks and flag unreliable questions
- Support small studies of how people evaluate model outputs
Requirements
- Zero to five years of professional experience
- Experience in QA, test engineering, assessment design, data analysis, ML evaluation or a similar field
- Experience writing test cases, exam questions, rubrics or evaluation tasks
- Ability to use spreadsheets or Python to organize and analyze results
- Clear, professional written communication
Preferred Qualifications
- Experience with evaluation tools such as Inspect, lm-eval-harness, HELM or promptfoo
- Experience with model comparisons, or user and rating studies
Benefits
- Compensation of $30 to $60 per hour, depending on project, field and experience
- Payment for hours worked
- Fully remote; projects typically run from one week to several months
What we look for
- Zero to five years of professional experience
- Experience in QA, test engineering, assessment design, data analysis, ML evaluation or a similar field
- Experience writing test cases, exam questions, rubrics or evaluation tasks
- Ability to use spreadsheets or Python to organize and analyze results
- Clear, professional written communication
- Experience with evaluation tools such as Inspect, lm-eval-harness, HELM or promptfoo
- Experience with model comparisons, or user and rating studies
PythonExcelDocumentationData AnalysisML Evaluation
SponsorWake up to a shortlist, not a search results page.
NeverApply scores every new listing against your CV, salary floor and visa. A handful of real matches by morning.
Get your daily matchesSimilar jobsbased on title, category and scope
SponsorWake up to a shortlist, not a search results page.
NeverApply scores every new listing against your CV, salary floor and visa. A handful of real matches by morning.
Get your daily matchesExpert Talent Pool, Evals and Benchmarks (Associate)Clera · free account, 30 seconds