Oracle
At Oracle Analytics, we are building the next generation of enterprise AI products to enable intelligent data analysis at scale. Leveraging our foundational strengths in data management and enterprise software applications, we advance our platforms and applications by deeply embedding cutting‑edge agentic AI, generative AI, and innovations in machine learning and optimization.
We are looking for a Senior Applied Scientist (PhD preferred) to innovate in AI agent evaluation and user experience (UX) and to advance machine learning for structured data at enterprise scale. The role involves defining evaluation frameworks for multi‑step, tool‑using agents; designing user‑centric studies and telemetry; building robust metrics that connect agent behavior to measurable user and business outcomes; and developing, productionizing ML methods for structured/tabular data—including forecasting, anomaly detection, and causal insights that power analytic workflows and agent decision‑making. You’ll partner closely with research engineers, product/UX, and platform teams to ship reliable, safe, and delightful agent experiences into production.
Responsibilities
Perform end‑to‑end agent evaluation programs: establish task taxonomies, gold standards, and reproducible scenarios for tool‑using and planning agents across BI, SQL, and data preparation workflows; design rubric‑based and pairwise evaluations to measure correctness, grounding, and traceability.
Stay current with research and translate advances into production differentiators; mentor teammates and contribute to a culture of scientific rigor and impact.
Minimum Qualifications
PhD in Computer Science, Machine Learning, Human‑Computer Interaction, Statistics, Electrical Engineering, or related field with focus relevant to AI agents, evaluation/UX, or structured data ML.
Experience (industry or applied research) building and evaluating ML systems, including agentic workflows, UX research for AI products, and structured/tabular ML in production.
Demonstrated expertise in human/AI evaluation: rubric and study design, telemetry and instrumentation, A/B testing, and statistical analysis; ability to connect evaluation metrics to product outcomes.
Strong background in structured data ML: feature engineering, time series forecasting, anomaly detection, causal inference, and robustness for enterprise datasets; proficiency with SQL and data warehousing concepts.
Proficient in Python and modern ML stacks (e.g., PyTorch/JAX) and libraries for evaluation and experimentation; solid software engineering practices, experiment tracking, and reproducibility discipline.
Track record of publications in top venues (e.g., NeurIPS, ICML, NAACL, I, ACL, CHI) or equivalent evidence of impact.
Preferred Qualifications
Experience building scalable evaluation platforms for tool‑using agents.
Expertise in UX research methods for AI.
Knowledge of planning and tool‑use policies.
Familiarity with efficiency and serving: batching, KV cache management, quantization/PEFT for cost/latency trade‑offs; observability and alerting for agent behavior drifts.
Experience integrating safety/guardrails, policy enforcement, and privacy‑preserving telemetry into production workflows aligned with enterprise compliance.
Comfortable collaborating across research, engineering, product, and legal/compliance; excellent communication skills to explain methods and results to technical and non‑technical stakeholders.
Benefits
Medical, dental, and vision insurance, including expert medical opinion.
Short‑term and long‑term disability.
Life insurance and AD&DD.
Supplemental life insurance (Employee/Spouse/Child).
Health care and dependent care Flexible Spending Accounts.
Pre‑tax commuter and parking benefits.
401(k) Savings and Investment Plan with company match.
Paid time off: Flexible Vacation; accrued vacations based on hours worked.
11 paid holidays.
Paid sick leave: 72 hours upon hire, refreshing each calendar year.
Paid parental leave, adoption assistance.
Employee Stock Purchase Plan.
Financial planning and group legal.
Voluntary benefits including auto, homeowner, and pet insurance.
We are an equal‑employment‑opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability, or protected veteran status. Oracle will consider qualified applicants with arrest and conviction records pursuant to applicable law.
Oracle is committed to including people with disabilities at all stages of the employment process. If you require accessibility assistance or accommodation, let us know at accommodation-request_mb@oracle.com or call +1 888 404 2494 in the United States.
#J-18808-Ljbffr
Responsibilities
Perform end‑to‑end agent evaluation programs: establish task taxonomies, gold standards, and reproducible scenarios for tool‑using and planning agents across BI, SQL, and data preparation workflows; design rubric‑based and pairwise evaluations to measure correctness, grounding, and traceability.
Stay current with research and translate advances into production differentiators; mentor teammates and contribute to a culture of scientific rigor and impact.
Minimum Qualifications
PhD in Computer Science, Machine Learning, Human‑Computer Interaction, Statistics, Electrical Engineering, or related field with focus relevant to AI agents, evaluation/UX, or structured data ML.
Experience (industry or applied research) building and evaluating ML systems, including agentic workflows, UX research for AI products, and structured/tabular ML in production.
Demonstrated expertise in human/AI evaluation: rubric and study design, telemetry and instrumentation, A/B testing, and statistical analysis; ability to connect evaluation metrics to product outcomes.
Strong background in structured data ML: feature engineering, time series forecasting, anomaly detection, causal inference, and robustness for enterprise datasets; proficiency with SQL and data warehousing concepts.
Proficient in Python and modern ML stacks (e.g., PyTorch/JAX) and libraries for evaluation and experimentation; solid software engineering practices, experiment tracking, and reproducibility discipline.
Track record of publications in top venues (e.g., NeurIPS, ICML, NAACL, I, ACL, CHI) or equivalent evidence of impact.
Preferred Qualifications
Experience building scalable evaluation platforms for tool‑using agents.
Expertise in UX research methods for AI.
Knowledge of planning and tool‑use policies.
Familiarity with efficiency and serving: batching, KV cache management, quantization/PEFT for cost/latency trade‑offs; observability and alerting for agent behavior drifts.
Experience integrating safety/guardrails, policy enforcement, and privacy‑preserving telemetry into production workflows aligned with enterprise compliance.
Comfortable collaborating across research, engineering, product, and legal/compliance; excellent communication skills to explain methods and results to technical and non‑technical stakeholders.
Benefits
Medical, dental, and vision insurance, including expert medical opinion.
Short‑term and long‑term disability.
Life insurance and AD&DD.
Supplemental life insurance (Employee/Spouse/Child).
Health care and dependent care Flexible Spending Accounts.
Pre‑tax commuter and parking benefits.
401(k) Savings and Investment Plan with company match.
Paid time off: Flexible Vacation; accrued vacations based on hours worked.
11 paid holidays.
Paid sick leave: 72 hours upon hire, refreshing each calendar year.
Paid parental leave, adoption assistance.
Employee Stock Purchase Plan.
Financial planning and group legal.
Voluntary benefits including auto, homeowner, and pet insurance.
We are an equal‑employment‑opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability, or protected veteran status. Oracle will consider qualified applicants with arrest and conviction records pursuant to applicable law.
Oracle is committed to including people with disabilities at all stages of the employment process. If you require accessibility assistance or accommodation, let us know at accommodation-request_mb@oracle.com or call +1 888 404 2494 in the United States.
#J-18808-Ljbffr