Call For Referral
Data Scientist – Remote (Part-Time | $100 –$120/hr)
Call For Referral, San Francisco, California, United States, 94199
Data Scientist (AI Task Evaluation & Statistical Analysis Specialist)
Hourly Contract | Part-Time Remote | $100 –$120 per hour
1. About the Role
Mercor is partnering with a
leading AI research lab
to hire experienced
Data Scientists
specializing in
AI task evaluation and statistical analysis .
In this role, you will conduct
comprehensive failure analysis
on AI agent performance across finance-sector tasks — identifying systemic patterns, diagnosing performance bottlenecks, and improving model evaluation frameworks.
You’ll work closely with AI engineers and research analysts to transform raw evaluation data into actionable insights, strengthening the quality, fairness, and reliability of large-scale AI systems.
2. Key Responsibilities
Statistical Failure Analysis:
Identify recurring patterns in AI agent failures across task components (prompts, rubrics, file types, tags, etc.).
Root Cause Analysis:
Determine whether issues stem from
task design ,
rubric clarity ,
file complexity , or
agent limitations .
Dimensional Analysis:
Examine performance variations across
finance sub-domains , file structures, and evaluation criteria.
Visualization & Reporting:
Build
dashboards and analytical reports
that highlight edge cases, performance clusters, and opportunities for improvement.
Framework Enhancement:
Recommend refinements to
rubric design, evaluation metrics, and task structures
based on empirical findings.
Stakeholder Communication:
Present key insights to
data labeling teams, ML engineers, and research collaborators .
3. Required Qualifications
Strong foundation in
statistical analysis, hypothesis testing, and pattern recognition .
Proficiency in
Python
(pandas, scipy, matplotlib/seaborn) or
R
for data analysis.
Hands‑on experience with
exploratory data analysis (EDA)
and
feature interpretation .
Understanding of
AI/ML evaluation methodologies
and
LLM performance metrics .
Skilled in using
Excel ,
SQL , and
data visualization tools
(e.g., Tableau, Looker).
4. Preferred Qualifications
Experience with
AI/ML model evaluation
or
quality assurance pipelines .
Background in
finance
or interest in learning financial domain structures.
Familiarity with
benchmark datasets ,
failure mode analysis , and
evaluation frameworks .
2–4 years
of relevant professional experience in data science, analytics, or applied statistics.
5. More About the Opportunity
Commitment:
Part‑time, 20–25 hours/week
Schedule:
Fully remote and asynchronous — work on your own time
Duration:
1–2 months, with strong potential for extension
Start Date:
Immediate
6. Compensation & Contract Terms
Hourly Rate:
$100–$120/hour (based on experience and region)
Classification:
Independent Contractor (via Mercor)
Payments:
Weekly via
Stripe Connect
for approved work
⚡
PS: Mercor reviews applications daily. Please complete your interview and onboarding steps to be considered for this opportunity.
⚡
#J-18808-Ljbffr
leading AI research lab
to hire experienced
Data Scientists
specializing in
AI task evaluation and statistical analysis .
In this role, you will conduct
comprehensive failure analysis
on AI agent performance across finance-sector tasks — identifying systemic patterns, diagnosing performance bottlenecks, and improving model evaluation frameworks.
You’ll work closely with AI engineers and research analysts to transform raw evaluation data into actionable insights, strengthening the quality, fairness, and reliability of large-scale AI systems.
2. Key Responsibilities
Statistical Failure Analysis:
Identify recurring patterns in AI agent failures across task components (prompts, rubrics, file types, tags, etc.).
Root Cause Analysis:
Determine whether issues stem from
task design ,
rubric clarity ,
file complexity , or
agent limitations .
Dimensional Analysis:
Examine performance variations across
finance sub-domains , file structures, and evaluation criteria.
Visualization & Reporting:
Build
dashboards and analytical reports
that highlight edge cases, performance clusters, and opportunities for improvement.
Framework Enhancement:
Recommend refinements to
rubric design, evaluation metrics, and task structures
based on empirical findings.
Stakeholder Communication:
Present key insights to
data labeling teams, ML engineers, and research collaborators .
3. Required Qualifications
Strong foundation in
statistical analysis, hypothesis testing, and pattern recognition .
Proficiency in
Python
(pandas, scipy, matplotlib/seaborn) or
R
for data analysis.
Hands‑on experience with
exploratory data analysis (EDA)
and
feature interpretation .
Understanding of
AI/ML evaluation methodologies
and
LLM performance metrics .
Skilled in using
Excel ,
SQL , and
data visualization tools
(e.g., Tableau, Looker).
4. Preferred Qualifications
Experience with
AI/ML model evaluation
or
quality assurance pipelines .
Background in
finance
or interest in learning financial domain structures.
Familiarity with
benchmark datasets ,
failure mode analysis , and
evaluation frameworks .
2–4 years
of relevant professional experience in data science, analytics, or applied statistics.
5. More About the Opportunity
Commitment:
Part‑time, 20–25 hours/week
Schedule:
Fully remote and asynchronous — work on your own time
Duration:
1–2 months, with strong potential for extension
Start Date:
Immediate
6. Compensation & Contract Terms
Hourly Rate:
$100–$120/hour (based on experience and region)
Classification:
Independent Contractor (via Mercor)
Payments:
Weekly via
Stripe Connect
for approved work
⚡
PS: Mercor reviews applications daily. Please complete your interview and onboarding steps to be considered for this opportunity.
⚡
#J-18808-Ljbffr