Datadog
Staff Applied Scientist - Observability Data Platform
Datadog, California, Missouri, United States, 65018
Overview
Staff Applied Scientist - Observability Data Platform at Datadog. The Observability Data Platform (ODP) powers the core of Datadog’s telemetry systems, handling exabytes of multimodal observability data. As AI agents become first-class consumers of telemetry, ODP is evolving to meet their demands — scaling with explosive data growth, exposing new query mechanisms, rethinking how telemetry is stored, transformed, and served, and enforcing guardrails that ensure security and reliability. Our team is building an intelligent control plane for production systems, moving beyond passive monitoring to enable AI agents to safely take action in live environments. This involves integrating techniques from symbolic reasoning, formal methods, and generative AI. We are looking for an experienced Staff Applied Scientist with a background spanning systems engineering, AI, and formal reasoning. You have expertise in areas such as causal modeling, generative simulation, runtime verification, or reinforcement learning, and are motivated to apply these skills to build reliable systems. Datadog values collaboration and innovation. We operate as a hybrid workplace to support work-life harmony. What You’ll Do
Design and prototype intelligent systems for AI-native observability, including cost-aware agent orchestration, adaptive query execution, and self-optimizing system components. Lead efforts to apply reinforcement learning, search, or hybrid approaches to infrastructure-level decision-making (autoscaling, scheduling, load shaping). Collaborate with AI researchers and platform engineers to design experimentation loops and verifiers that guide LLM outputs using runtime metrics and formal models. Explore emerging paradigms like AI compilers, “programming after code,” and runtime-aware prompt engineering to inform Datadog’s infrastructure and product design. Help define the direction of BitsEvolve — Datadog’s optimization agent that uses LLMs and evolutionary search to discover code improvements, optimize GPU kernels, and tune configurations to improve performance. Partner with product teams and platform stakeholders to translate scientific advances into measurable improvements in cost, performance, and observability depth. Who You Are
You have a BS/MS/PhD in a scientific field or equivalent experience. You have 8+ years of experience in systems engineering, database internals, or infrastructure research, including hands-on experience in a production environment. You have a strong software engineering foundation, ideally in C++, Rust, Go, or Python, and are comfortable writing performant, maintainable code. You have deep expertise in at least one of the following: query optimization, data center scheduling, compiler design, reinforcement learning, or distributed systems design. You have experience applying search, planning, or learning techniques to real-world optimization problems. You are excited by systems that learn, adapt, and improve over time using feedback from runtime metrics and human-defined objectives. You are hypothesis-driven and enjoy designing experiments and evaluation loops, whether through simulations, benchmarks, or live systems. You thrive in ambiguity, enjoy reading papers and building prototypes, and want to help shape the future of infrastructure in the AI era. You enjoy collaborating across research, engineering, and product to bring scientific insights to practical outcomes. Benefits and Growth
Get to build tools for software engineers, and use the tools we build to accelerate development. Have a lot of influence on product direction and impact on the business. Work with skilled, knowledgeable, and kind teammates who are happy to teach and learn. Competitive global benefits. Continuous professional development. Datadog offers a competitive salary and equity package, and may include variable compensation. The estimated yearly salary for this role ranges from $234,000 to $300,000 USD, with actual compensation based on skills and experience. Benefits include healthcare, parental planning, mental health benefits, 401(k) with match, paid time off, fitness reimbursements, and employee stock purchase plan. About Datadog
Datadog (NASDAQ: DDOG) is a global SaaS business solving complexity in the cloud age by enabling digital transformation, cloud migration, and infrastructure monitoring. We champion professional development, diversity of thought, innovation, and work excellence. Datadog is an equal opportunity employer and details can be found in our Candidate Legal Notices and Privacy and AI Guidelines. If you need accessibility accommodations, please contact us through the provided form.
#J-18808-Ljbffr
Staff Applied Scientist - Observability Data Platform at Datadog. The Observability Data Platform (ODP) powers the core of Datadog’s telemetry systems, handling exabytes of multimodal observability data. As AI agents become first-class consumers of telemetry, ODP is evolving to meet their demands — scaling with explosive data growth, exposing new query mechanisms, rethinking how telemetry is stored, transformed, and served, and enforcing guardrails that ensure security and reliability. Our team is building an intelligent control plane for production systems, moving beyond passive monitoring to enable AI agents to safely take action in live environments. This involves integrating techniques from symbolic reasoning, formal methods, and generative AI. We are looking for an experienced Staff Applied Scientist with a background spanning systems engineering, AI, and formal reasoning. You have expertise in areas such as causal modeling, generative simulation, runtime verification, or reinforcement learning, and are motivated to apply these skills to build reliable systems. Datadog values collaboration and innovation. We operate as a hybrid workplace to support work-life harmony. What You’ll Do
Design and prototype intelligent systems for AI-native observability, including cost-aware agent orchestration, adaptive query execution, and self-optimizing system components. Lead efforts to apply reinforcement learning, search, or hybrid approaches to infrastructure-level decision-making (autoscaling, scheduling, load shaping). Collaborate with AI researchers and platform engineers to design experimentation loops and verifiers that guide LLM outputs using runtime metrics and formal models. Explore emerging paradigms like AI compilers, “programming after code,” and runtime-aware prompt engineering to inform Datadog’s infrastructure and product design. Help define the direction of BitsEvolve — Datadog’s optimization agent that uses LLMs and evolutionary search to discover code improvements, optimize GPU kernels, and tune configurations to improve performance. Partner with product teams and platform stakeholders to translate scientific advances into measurable improvements in cost, performance, and observability depth. Who You Are
You have a BS/MS/PhD in a scientific field or equivalent experience. You have 8+ years of experience in systems engineering, database internals, or infrastructure research, including hands-on experience in a production environment. You have a strong software engineering foundation, ideally in C++, Rust, Go, or Python, and are comfortable writing performant, maintainable code. You have deep expertise in at least one of the following: query optimization, data center scheduling, compiler design, reinforcement learning, or distributed systems design. You have experience applying search, planning, or learning techniques to real-world optimization problems. You are excited by systems that learn, adapt, and improve over time using feedback from runtime metrics and human-defined objectives. You are hypothesis-driven and enjoy designing experiments and evaluation loops, whether through simulations, benchmarks, or live systems. You thrive in ambiguity, enjoy reading papers and building prototypes, and want to help shape the future of infrastructure in the AI era. You enjoy collaborating across research, engineering, and product to bring scientific insights to practical outcomes. Benefits and Growth
Get to build tools for software engineers, and use the tools we build to accelerate development. Have a lot of influence on product direction and impact on the business. Work with skilled, knowledgeable, and kind teammates who are happy to teach and learn. Competitive global benefits. Continuous professional development. Datadog offers a competitive salary and equity package, and may include variable compensation. The estimated yearly salary for this role ranges from $234,000 to $300,000 USD, with actual compensation based on skills and experience. Benefits include healthcare, parental planning, mental health benefits, 401(k) with match, paid time off, fitness reimbursements, and employee stock purchase plan. About Datadog
Datadog (NASDAQ: DDOG) is a global SaaS business solving complexity in the cloud age by enabling digital transformation, cloud migration, and infrastructure monitoring. We champion professional development, diversity of thought, innovation, and work excellence. Datadog is an equal opportunity employer and details can be found in our Candidate Legal Notices and Privacy and AI Guidelines. If you need accessibility accommodations, please contact us through the provided form.
#J-18808-Ljbffr