Expert Talent Pool, Evals and Benchmarks (Associate)

Clera

WorldwideFully remoteContractEntry Level, JuniorInformation Technology
USD 30–60 an hourPosted 4h ago

Why we think you can apply

Hiring model
Open worldwide · the employer states no country restriction
Work from
Anywhere, the UAE included

Posted 4 hours ago · apply link checked 4 hours ago

About the role

Responsibilities

  • Write questions and tasks that test specific model capabilities
  • Develop rubrics defining strong, partial and weak answers
  • Create reference answers and edge-case scenarios
  • Grade model outputs against rubrics and explain each score
  • Analyze incorrect answers, group error patterns and document findings
  • Compare multiple models using the same prompts
  • Validate benchmarks and flag unreliable questions
  • Support small studies of how people evaluate model outputs

Requirements

  • Zero to five years of professional experience
  • Experience in QA, test engineering, assessment design, data analysis, ML evaluation or a similar field
  • Experience writing test cases, exam questions, rubrics or evaluation tasks
  • Ability to use spreadsheets or Python to organize and analyze results
  • Clear, professional written communication

Preferred Qualifications

  • Experience with evaluation tools such as Inspect, lm-eval-harness, HELM or promptfoo
  • Experience with model comparisons, or user and rating studies

Benefits

  • Compensation of $30 to $60 per hour, depending on project, field and experience
  • Payment for hours worked
  • Fully remote; projects typically run from one week to several months

What we look for

  • Zero to five years of professional experience
  • Experience in QA, test engineering, assessment design, data analysis, ML evaluation or a similar field
  • Experience writing test cases, exam questions, rubrics or evaluation tasks
  • Ability to use spreadsheets or Python to organize and analyze results
  • Clear, professional written communication
  • Experience with evaluation tools such as Inspect, lm-eval-harness, HELM or promptfoo
  • Experience with model comparisons, or user and rating studies
PythonExcelDocumentationData AnalysisML Evaluation
NeverApplySponsor

Wake up to a shortlist, not a search results page.

NeverApply scores every new listing against your CV, salary floor and visa. A handful of real matches by morning.

Get your daily matches
Similar jobsbased on title, category and scope
Expert Talent Pool, Evals and Benchmarks (Associate)free account, 30 seconds

Every new role, on Telegram.

Members see each new UAE role first. A day later it lands in the channel, where 6,800 people follow.

FreeNo spamUpdated hourly
Or show me jobs that hire in the UAE

Remote Jobs in Dubai UAE

6,800+ followers

Get alerts on Telegram