Amazon
Overview
We are looking for an innovative Applied Scientist to join the Books Store team. This role will start with a focus on our centralized auto-evaluation and guardrail platform for AI-powered features across Books. This role is critical in designing and implementing scalable, science-driven evaluation solutions that ensure consistent quality standards across features such as summarization, recommendations, character insights, and more. Responsibilities
Design, prototype, and productionize auto-evaluation methods (e.g., for faithfulness, coherence, tone, personalization, safety) to assess LLM-generated content at scale. Collaborate with feature teams to adapt and extend evaluation tools to meet evolving feature-specific needs. Develop scientifically rigorous and efficient experimentation frameworks to compare prompt, model, or context configurations. Drive prompt optimization strategies using evaluator feedback, error clustering, and automated suggestion mechanisms. Contribute to defining and implementing the AI Bar Raiser (AIBR) process for design and launch reviews of AI features. Advance the state-of-the-art in LLM evaluation through internal innovation or external research contributions. Basic Qualifications
PhD, or Master\u2019s degree and 6+ years of applied research experience 4+ years of applied research experience 3+ years of building machine learning models for business application experience Experience programming in Java, C++, Python or related language Preferred Qualifications
Experience with popular deep learning frameworks such as MxNet and Tensor Flow. Experience with large scale distributed systems such as Hadoop, Spark etc. Experience with neural deep learning methods and machine learning Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status. Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner. Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $150,400/year in our lowest geographic market up to $260,000/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, please visit https://www.aboutamazon.com/workplace/employee-benefits. This position will remain posted until filled. Applicants should apply via our internal or external career site. Company - Amazon.com Services LLC Job ID: A3068688 Seniority level
Mid-Senior level Employment type
Full-time Job function
Research, Science, and Engineering Industries: Software Development Referrals increase your chances of interviewing at Amazon by 2x Get notified about new Store Assistant jobs in Seattle, WA.
#J-18808-Ljbffr
We are looking for an innovative Applied Scientist to join the Books Store team. This role will start with a focus on our centralized auto-evaluation and guardrail platform for AI-powered features across Books. This role is critical in designing and implementing scalable, science-driven evaluation solutions that ensure consistent quality standards across features such as summarization, recommendations, character insights, and more. Responsibilities
Design, prototype, and productionize auto-evaluation methods (e.g., for faithfulness, coherence, tone, personalization, safety) to assess LLM-generated content at scale. Collaborate with feature teams to adapt and extend evaluation tools to meet evolving feature-specific needs. Develop scientifically rigorous and efficient experimentation frameworks to compare prompt, model, or context configurations. Drive prompt optimization strategies using evaluator feedback, error clustering, and automated suggestion mechanisms. Contribute to defining and implementing the AI Bar Raiser (AIBR) process for design and launch reviews of AI features. Advance the state-of-the-art in LLM evaluation through internal innovation or external research contributions. Basic Qualifications
PhD, or Master\u2019s degree and 6+ years of applied research experience 4+ years of applied research experience 3+ years of building machine learning models for business application experience Experience programming in Java, C++, Python or related language Preferred Qualifications
Experience with popular deep learning frameworks such as MxNet and Tensor Flow. Experience with large scale distributed systems such as Hadoop, Spark etc. Experience with neural deep learning methods and machine learning Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status. Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner. Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $150,400/year in our lowest geographic market up to $260,000/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, please visit https://www.aboutamazon.com/workplace/employee-benefits. This position will remain posted until filled. Applicants should apply via our internal or external career site. Company - Amazon.com Services LLC Job ID: A3068688 Seniority level
Mid-Senior level Employment type
Full-time Job function
Research, Science, and Engineering Industries: Software Development Referrals increase your chances of interviewing at Amazon by 2x Get notified about new Store Assistant jobs in Seattle, WA.
#J-18808-Ljbffr