Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting)

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting) based in Canada. This is a short-term, structured opportunity for experienced business professionals to help evaluate the quality of next-generation AI models. You will complete one realistic professional task based on your domain expertise and then assess five AI-generated attempts at that same task. Your professional judgment will serve as the benchmark for evaluating accuracy, reasoning, clarity, and practical relevance. You will provide evidence-based scores and detailed rationales that can directly influence future AI model development. The study is designed for practitioners who understand how work is actually performed in real business environments. It offers a focused way to contribute your expertise to AI research without requiring a long-term commitment. The engagement is fully remote, flexible, and expected to require approximately 8–12 hours of your time. Accountabilities Sign a confidentiality agreement before receiving project materials and ensure all study information is handled appropriately. Complete one realistic, domain-specific professional task using the source materials provided. Review five AI-generated responses to the same task and rank them from 1 to 5 with no ties. Assign each model response a score from 0–100 based on the quality of the work and your professional judgment. Write clear, specific, evidence-based rationales that reference relevant portions of each response to support your evaluation. Assess the accuracy, reasoning, practical applicability, and overall quality of the model-generated work within your professional domain. Provide structured feedback on the task itself and the overall study design to help improve future evaluation processes. Complete all work independently without using LLMs or other AI assistants during task completion, evaluation, scoring, or rationale writing. Requirements 5+ years of hands-on professional experience in Human Resources or Accounting. Current or recent professional practice in your field, with sufficient experience to evaluate work according to real-world professional standards. Strong ability to assess professional work critically and distinguish high-quality reasoning from flawed, incomplete, or unsupported conclusions. Excellent written communication skills, with the ability to produce concise, specific, and evidence-grounded explanations for your judgments. Strong attention to detail, analytical thinking, and professional judgment. Ability to work reliably within a short turnaround period and complete a structured evaluation independently. Comfortable working without a predefined rubric or “golden answer,” using your own professional expertise as the primary basis for evaluation. Must be legally able to participate as an independent contractor in Canada. Must not have previously worked on Project Alchemy in any capacity or accessed its source-world data, as prior exposure would invalidate the study. Candidates requiring H-1B or STEM OPT sponsorship are not eligible for this engagement. Benefits Fully remote, independent-contractor engagement that can be completed on your own schedule. Estimated commitment of 8–12 hours for the one-time evaluation study. Opportunity to contribute directly to the development and evaluation of advanced AI systems. Apply your professional expertise to realistic business scenarios without a long-term production commitment. Competitive payment for completed work, with weekly payments via Stripe or Wise. Flexible project structure; engagements may be extended, shortened, or concluded early depending on project needs and performance. No access to confidential or proprietary information belonging to your current or former employers, clients, or institutions. Opportunity to earn up to $400 per successful referral, subject to applicable restrictions. How Jobgether Works We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team. We appreciate your interest and wish you the best! Why Apply Through Jobgether? Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time. We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.

Evaluate the quality of next-generation AI models by completing professional tasks and scoring AI-generated responses. Provide detailed, evidence-based rationales to benchmark accuracy, reasoning, and practical relevance.

Requires 5+ years of professional experience in Human Resources or Accounting with current practice in the field. Must be legally able to work as an independent contractor in Canada and possess strong critical analysis skills.

• Fully Remote Work • Flexible Schedule • Competitive Payment • Weekly Payments via Stripe or Wise • Referral Bonus

• written communicationSouhaitée
• Souci du détailSouhaitée
• ComptabilitéSouhaitée

Relevées dans la description du poste. Confirmez les exigences importantes ci-dessus.

• AI Model Evaluation
• Human Resources
• Accounting
• Analytical Thinking
• Professional Judgment
• Written Communication
• Critical Assessment
• Attention To Detail

• Data & Analytics
• Human Resources
• Finance & Accounting
• Administrative
• Technology

Expérience minimale

5+ ans

Langue de l’offre

anglais

Heures de travail

12 heures par semaine

Niveau d’expérience

Mid-Senior level

Mode de candidature

La candidature directe est offerte