AI QA Trainer

Responsibilities

• Converse with models on real-world scenarios and evaluation prompts
• Verify factual accuracy and logical soundness
• Design and run test plans and regression suites
• Build clear rubrics and pass/fail criteria
• Capture reproducible error traces with root-cause hypotheses
• Suggest improvements to prompt engineering, guardrails, and evaluation metrics
• Partner on adversarial red-teaming, automation, and quality dashboarding

Requirements

• Experience in model evaluation, LLM safety, and prompt robustness
• Expertise in hallucination detection and factual consistency
• Knowledge of prompt-injection and jailbreak resistance
• Ability to perform bias/fairness audits and grounding verification
• Proficiency in test automation using Python or SQL
• Strong metacognitive communication skills for high-signal bug reporting

Preferred Qualifications

• Bachelor’s, Master’s, or PhD in Computer Science, Data Science, Computational Linguistics, or Statistics
• Experience shipping QA for ML/AI systems
• Experience with test automation frameworks like PyTest
• Hands-on work with LLM eval tooling such as OpenAI Evals, RAG evaluators, or W&B

About the Company

We are seeking AI QA experts to help harden model reasoning and reliability, ensuring large-scale language models evolve into enterprise-grade, safe, and reliable platforms.