The Rundown AI, Inc.
Senior Software Engineer — AI/ML
The Rundown AI, Inc., San Francisco, California, United States
As a Software Engineer on the Evaluation Engineering team, you'll build systems to power large-scale AI workloads for top tier AI research labs. You’ll work closely with other engineers, product managers, and field team members to ensure that Snorkel’s frontier AI datasets meet and surpass the capabilities of the most advanced foundation models. You will launch agents into production to accomplish long running reasoning and verification tasks. The Evaluation Engineering team owns critical systems for building high quality post-training and benchmark datasets, integrating with the latest foundation model technology to push the frontier of models used globally across coding, mathematics, law, medicine, and other advanced domains.
Main Responsibilities
Own the architecture, design, development, and operations of large-scale systems designed for AI/ML tasks including distributed compute systems, data management systems, data engineering workflow systems, and end user experiences
Recognize and act on opportunities to integrate the latest agentic and foundation model technologies to power eval workflows
Prototype, optimize, and maintain scalable back-end services that will power new foundation model development tools
Design extensible and testable interfaces between internal services including the underlying storage and data models
Be an engaged team player in a customer-focused cross-functional environment where you will feel excited to take on whatever is most impactful for the company and product
Work a hybrid schedule with 3 days per week in one of our offices in San Francisco or Redwood City
Required Qualifications
4+ years experience in delivering AI/ML systems and services in a production setting for cloud-native applications
Experience with distributed compute frameworks
Experience with the modern AI stack, including improving LLM applications through evals, prompting, and agent scaffolding
Ability to design and build efficient data storage, compute, and retrieval systems for AI/ML tasks
Strong communication and coding skills with emphasis on designing for scale and robustness
Experience owning the delivery of large multi-person projects
Preferred Qualifications
8+ years of professional software engineering experience
Experience with architecting and developing production web-scale systems (monitoring, telemetry, performance, reliability, triage and debug)
Strong development and debugging skills in Python
Experience with expert data annotation projects
Experience developing evaluations and environments for complex multi-turn and multi-tool AI systems
#J-18808-Ljbffr
Main Responsibilities
Own the architecture, design, development, and operations of large-scale systems designed for AI/ML tasks including distributed compute systems, data management systems, data engineering workflow systems, and end user experiences
Recognize and act on opportunities to integrate the latest agentic and foundation model technologies to power eval workflows
Prototype, optimize, and maintain scalable back-end services that will power new foundation model development tools
Design extensible and testable interfaces between internal services including the underlying storage and data models
Be an engaged team player in a customer-focused cross-functional environment where you will feel excited to take on whatever is most impactful for the company and product
Work a hybrid schedule with 3 days per week in one of our offices in San Francisco or Redwood City
Required Qualifications
4+ years experience in delivering AI/ML systems and services in a production setting for cloud-native applications
Experience with distributed compute frameworks
Experience with the modern AI stack, including improving LLM applications through evals, prompting, and agent scaffolding
Ability to design and build efficient data storage, compute, and retrieval systems for AI/ML tasks
Strong communication and coding skills with emphasis on designing for scale and robustness
Experience owning the delivery of large multi-person projects
Preferred Qualifications
8+ years of professional software engineering experience
Experience with architecting and developing production web-scale systems (monitoring, telemetry, performance, reliability, triage and debug)
Strong development and debugging skills in Python
Experience with expert data annotation projects
Experience developing evaluations and environments for complex multi-turn and multi-tool AI systems
#J-18808-Ljbffr