Khan Academy
Senior Platform Engineer, AI Evaluation (24 months fixed-term)
Khan Academy, Mountain View, California, us, 94039
Mountain View, CA / Remote (Continental US + Hawaii + Canada Only)
ABOUT KHAN ACADEMY Khan Academy is a nonprofit with the mission to deliver a free, world‑class education to anyone, anywhere. Our proven learning platform offers free, high‑quality supplemental learning content and practice that cover Pre‑K - 12th grade and early college core academic subjects, focusing on math and science. We have over 181 million registered learners globally and are committed to improving learning outcomes for students worldwide, focusing on learners in historically under‑resourced communities.
OUR COMMUNITY Our students, teachers, and parents come from all walks of life, and so do we. Our team includes people from academia, traditional/non‑traditional education, big tech companies, and tiny startups. We hire great people from diverse backgrounds and experiences because it makes our company stronger. We value diversity, equity, inclusion, and belonging as necessary to achieve our mission and impact the communities we serve. We know that transforming education starts in‑house with learning about ourselves and our colleagues. We strive to be world‑class in investing in our people and commit to developing you as a professional.
THE ROLE We’re looking for an AI Platform Engineer to evolve and extend our internal evaluation framework for assessing the quality of our AI‑driven experiences at Khan Academy. This engineer will have worked with enough eval systems to quickly make sense of Khan's internal eval framework and recognize opportunities for improvement. This is largely a software development role, but domain experience with AI eval is essential for appreciating the hill‑climbing and data science workflows we need to support. Soft skills will be important for gathering internal requirements, getting buy‑in for changes, and then developing documentation and training materials. You’ll work closely with ML data engineers and platform developers to help internal teams adopt an eval‑driven development process incorporating offline benchmark tests and online experiments.
As a
Platform Engineer focused on evaluation , you’ll be expected to:
Be fluent in the range of offline and online evaluation strategies, and when to apply the techniques over the lifecycle of development
Have intuitions about how to specify eval pipelines succinctly using declarative syntax
Understand the role of stratified datasets and ground truth labeling
Appreciate the range of eval scoring schemes from human raters to automated LLMs‑as‑judge
Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field
5 years of Software Engineering experience with 2+ of those years working on the evaluation of generative AI systems
Strong programming skills in Go, Python, SQL, and at least one data pipeline framework (e.g., Airflow, Dagster, Prefect)
Familiarity with the architecture of large language models and their industry‑standard APIs
Preferred
Experience with labeling platforms (e.g., Label Studio, Scale AI, Toloka) and human‑in‑the‑loop concerns such as rubric development and inter‑rater agreement
Exposure to MLOps practices such as model registry, feature store, or continuous evaluation
Background in education technology or other human‑centered AI applications
PERKS AND BENEFITS
Ample paid time off as needed – Your well‑being is a priority
8 pre‑scheduled Wellness Days in 2026 occurring on a Monday or a Friday for a 3‑day weekend boost
Remote‑first culture - that caters to your time zone, with open flexibility as needed, at times
Generous parental leave
An exceptional team that trusts you and gives you the freedom to do your best
The chance to put your talents towards a deeply meaningful mission and the opportunity to work on high‑impact products that are already defining the future of education
Opportunities to connect through affinity, ally, and social groups
and we offer all those other typical benefits as well: 401(k) + 4% matching & comprehensive insurance, including medical, dental, vision, and life
At Khan Academy we are committed to fair and equitable compensation practices, the well‑being of our employees, and our Khan community. This belief is why we have built out a robust Total Rewards package that includes competitive base salaries, and extensive benefits and perks to support physical, mental, and financial well‑being.
The target salary range for this position is $137,871 - $172,339 USD / $186,306 - $232,883 CAN. The pay range for this position is a general guideline only. The salary offered will depend on internal pay equity and the candidate’s relevant skills, experience, qualifications, and job market data.
We deeply understand and empathize with our users. We leverage user insights, research, and experience to build content, products, services, and experiences that our users trust and love. Our success is defined by the success of our learners and educators.
Take a stand As a company, we have conviction in our aspirational point of view of how education will evolve. The work we do is in service to moving towards that point of view. However, we also listen, learn and flex in the face of new data, and commit to evolving this point of view as the industry and our users evolve.
Work responsibly and sustainably We understand that achieving our audacious mission is a marathon, so we set realistic timelines and we focus on delivery that also links to the bigger picture. As a non‑profit, we are supported by the generosity of donors as well as strategic partners, and understand our responsibility to our finite resources. We spend every dollar as though it were our own. We are responsible for the impact we have on the world and to each other. We ensure our team and company stay healthy and financially sustainable.
Bring out the joy We are committed to making learning a joyful process. This informs what we build for our users and the culture we co‑create with our teammates, partners and donors.
Cultivate learning mindset We believe in the power of growth for learners and for ourselves. We constantly learn and teach to improve our offerings, ourselves, and our organization. We learn from our mistakes and aren’t afraid to fail. We don't let past failures or successes stop us from taking future bold action and achieving our goals.
Deliver wow We insist on high standards and deliver delightful, effective end‑to‑end experiences that our users can rely on. We choose to focus on fewer things — each of which aligns to our ambitious vision — so we can deliver high‑quality experiences that accelerate positive measurable learning with our strategic partners.
We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, gender, gender identity or expression, national origin, sexual orientation, age, citizenship, marital status, disability, or Veteran status. We value diversity, equity, and inclusion, and we encourage candidates from historically underrepresented groups to apply.
As part of this commitment, Khan Academy will ensure that persons with disabilities are provided reasonable accommodations for the hiring process. If reasonable accommodation is needed, please contact
careers@khanacademy.org .
#J-18808-Ljbffr
ABOUT KHAN ACADEMY Khan Academy is a nonprofit with the mission to deliver a free, world‑class education to anyone, anywhere. Our proven learning platform offers free, high‑quality supplemental learning content and practice that cover Pre‑K - 12th grade and early college core academic subjects, focusing on math and science. We have over 181 million registered learners globally and are committed to improving learning outcomes for students worldwide, focusing on learners in historically under‑resourced communities.
OUR COMMUNITY Our students, teachers, and parents come from all walks of life, and so do we. Our team includes people from academia, traditional/non‑traditional education, big tech companies, and tiny startups. We hire great people from diverse backgrounds and experiences because it makes our company stronger. We value diversity, equity, inclusion, and belonging as necessary to achieve our mission and impact the communities we serve. We know that transforming education starts in‑house with learning about ourselves and our colleagues. We strive to be world‑class in investing in our people and commit to developing you as a professional.
THE ROLE We’re looking for an AI Platform Engineer to evolve and extend our internal evaluation framework for assessing the quality of our AI‑driven experiences at Khan Academy. This engineer will have worked with enough eval systems to quickly make sense of Khan's internal eval framework and recognize opportunities for improvement. This is largely a software development role, but domain experience with AI eval is essential for appreciating the hill‑climbing and data science workflows we need to support. Soft skills will be important for gathering internal requirements, getting buy‑in for changes, and then developing documentation and training materials. You’ll work closely with ML data engineers and platform developers to help internal teams adopt an eval‑driven development process incorporating offline benchmark tests and online experiments.
As a
Platform Engineer focused on evaluation , you’ll be expected to:
Be fluent in the range of offline and online evaluation strategies, and when to apply the techniques over the lifecycle of development
Have intuitions about how to specify eval pipelines succinctly using declarative syntax
Understand the role of stratified datasets and ground truth labeling
Appreciate the range of eval scoring schemes from human raters to automated LLMs‑as‑judge
Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field
5 years of Software Engineering experience with 2+ of those years working on the evaluation of generative AI systems
Strong programming skills in Go, Python, SQL, and at least one data pipeline framework (e.g., Airflow, Dagster, Prefect)
Familiarity with the architecture of large language models and their industry‑standard APIs
Preferred
Experience with labeling platforms (e.g., Label Studio, Scale AI, Toloka) and human‑in‑the‑loop concerns such as rubric development and inter‑rater agreement
Exposure to MLOps practices such as model registry, feature store, or continuous evaluation
Background in education technology or other human‑centered AI applications
PERKS AND BENEFITS
Ample paid time off as needed – Your well‑being is a priority
8 pre‑scheduled Wellness Days in 2026 occurring on a Monday or a Friday for a 3‑day weekend boost
Remote‑first culture - that caters to your time zone, with open flexibility as needed, at times
Generous parental leave
An exceptional team that trusts you and gives you the freedom to do your best
The chance to put your talents towards a deeply meaningful mission and the opportunity to work on high‑impact products that are already defining the future of education
Opportunities to connect through affinity, ally, and social groups
and we offer all those other typical benefits as well: 401(k) + 4% matching & comprehensive insurance, including medical, dental, vision, and life
At Khan Academy we are committed to fair and equitable compensation practices, the well‑being of our employees, and our Khan community. This belief is why we have built out a robust Total Rewards package that includes competitive base salaries, and extensive benefits and perks to support physical, mental, and financial well‑being.
The target salary range for this position is $137,871 - $172,339 USD / $186,306 - $232,883 CAN. The pay range for this position is a general guideline only. The salary offered will depend on internal pay equity and the candidate’s relevant skills, experience, qualifications, and job market data.
We deeply understand and empathize with our users. We leverage user insights, research, and experience to build content, products, services, and experiences that our users trust and love. Our success is defined by the success of our learners and educators.
Take a stand As a company, we have conviction in our aspirational point of view of how education will evolve. The work we do is in service to moving towards that point of view. However, we also listen, learn and flex in the face of new data, and commit to evolving this point of view as the industry and our users evolve.
Work responsibly and sustainably We understand that achieving our audacious mission is a marathon, so we set realistic timelines and we focus on delivery that also links to the bigger picture. As a non‑profit, we are supported by the generosity of donors as well as strategic partners, and understand our responsibility to our finite resources. We spend every dollar as though it were our own. We are responsible for the impact we have on the world and to each other. We ensure our team and company stay healthy and financially sustainable.
Bring out the joy We are committed to making learning a joyful process. This informs what we build for our users and the culture we co‑create with our teammates, partners and donors.
Cultivate learning mindset We believe in the power of growth for learners and for ourselves. We constantly learn and teach to improve our offerings, ourselves, and our organization. We learn from our mistakes and aren’t afraid to fail. We don't let past failures or successes stop us from taking future bold action and achieving our goals.
Deliver wow We insist on high standards and deliver delightful, effective end‑to‑end experiences that our users can rely on. We choose to focus on fewer things — each of which aligns to our ambitious vision — so we can deliver high‑quality experiences that accelerate positive measurable learning with our strategic partners.
We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, gender, gender identity or expression, national origin, sexual orientation, age, citizenship, marital status, disability, or Veteran status. We value diversity, equity, and inclusion, and we encourage candidates from historically underrepresented groups to apply.
As part of this commitment, Khan Academy will ensure that persons with disabilities are provided reasonable accommodations for the hiring process. If reasonable accommodation is needed, please contact
careers@khanacademy.org .
#J-18808-Ljbffr