Checkmate
Prompt Engineer Data Science & Quality Analysis
Prompt Engineer Data Science & Quality Analysis
2 weeks ago Be among the first 25 applicants Get AI-powered advice on this job and more exclusive features. This range is provided by Checkmate. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more. Base pay range
$125.00/yr - $160.00/yr As a Prompt Engineer focused on Data Science and Quality Analysis, you'll design, test, and evaluate prompts for AI systems that interact with real-world restaurant data. You'll work cross-functionally to develop AI solutions that drive operational efficiency, improve data interpretation, and support smarter decision-making for restaurant operators.
Your work will directly shape how AI models perform in high-stakes, dynamic environments like order processing, reporting, support automation, and performance analysis.
Essential Job Functions:
Prompt Design & Evaluation: Develop, test, and refine prompts for tasks such as text generation, question answering, data classification, and structured data extraction to optimize Voice AI performance Data-Driven Analysis & Quality Measurement: Design evaluation frameworks and analyze prompt outputs using quantitative metrics, human-in-the-loop evaluation, and user feedback to identify improvement opportunities Experimentation & Iteration Conduct experiments to test prompt variations, measure their business and operational impact, and iterate to enhance accuracy, consistency, and safety Regression Testing & Compliance Build principled regression test suites using tools like LangFuse and Galileo to ensure prompts remain compliant and high-performing as models and use cases evolve. Collaboration Across Teams: Work closely with data science, product, legal, engineering, and operations teams to align prompt designs with business goals, operational workflows, and compliance requirements Model Adaptation & Strategy Development prompts across multiple LLMs (GPT, LLaMA, Gemini, and Checkmate's fine-tuned models), understanding model differences to optimize outputs effectively Team Leadership & Mentorship Lead a team of analysts focused on prompt evaluation and data quality analysis, guiding prioritization, experimentation, and reporting. Collaborate with ops teams for seamless deployment and feedback loops Research & Continuous Learning Stay up to date on emerging prompting techniques, LLM behaviors, evaluation frameworks, and AI safety practices to keep Checkmate's AI solutions best-in-class
Requirements
Strong analytical and data science skills, with hands-on experience in Python (pandas, NumPy, scikit-learn) Experience designing and conducting experiments and evaluations in applied AI or NLP contexts Proficiency in SQL and working with relational databases (e.g. MySQL, PostgreSQL, Oracle, MS SQL) Good understanding of data processing, quality measurement, and testing fundamentals Experience leading analyst or operations teams, with strong prioritization, mentorship, and collaboration skills Strong problem-solving mindset with a drive to explore, optimize, and automate workflows Excellent communication skills for presenting insights to technical and non-technical stakeholders Bachelor's degree in Data Science, Computer Science, Statistics, Engineering, or a related field Flexible to work US hours until at least 6 pm ET, with a strong remote setup
Preferred Qualifications
Experience with LLM evaluation and prompt engineering workflows Familiarity with tools like LangFuse and Galileo for prompt evaluation and analysis Knowledge of cloud platforms (AWS, GCP, Azure) and data pipeline tools Familiarity with machine learning concepts and NLP workflows Master's or PhD in Data Science, Statistics, Computer Science, or a related field
Benefits
Health Care Plan (Medical, Dental & Vision) Retirement Plan (401k) Life Insurance (Basic, Voluntary & AD&D) Flexible Paid Time Off Family Leave (Maternity, Paternity) Short Term & Long Term Disability Training & Development Work From Home Stock Option Plan
Seniority level
Seniority level Mid-Senior level Employment type
Employment type Full-time Job function
Job function Other Industries IT Services and IT Consulting Referrals increase your chances of interviewing at Checkmate by 2x Sign in to set job alerts for “Prompt Engineer” roles.
We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
#J-18808-Ljbffr
Prompt Engineer Data Science & Quality Analysis
2 weeks ago Be among the first 25 applicants Get AI-powered advice on this job and more exclusive features. This range is provided by Checkmate. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more. Base pay range
$125.00/yr - $160.00/yr As a Prompt Engineer focused on Data Science and Quality Analysis, you'll design, test, and evaluate prompts for AI systems that interact with real-world restaurant data. You'll work cross-functionally to develop AI solutions that drive operational efficiency, improve data interpretation, and support smarter decision-making for restaurant operators.
Your work will directly shape how AI models perform in high-stakes, dynamic environments like order processing, reporting, support automation, and performance analysis.
Essential Job Functions:
Prompt Design & Evaluation: Develop, test, and refine prompts for tasks such as text generation, question answering, data classification, and structured data extraction to optimize Voice AI performance Data-Driven Analysis & Quality Measurement: Design evaluation frameworks and analyze prompt outputs using quantitative metrics, human-in-the-loop evaluation, and user feedback to identify improvement opportunities Experimentation & Iteration Conduct experiments to test prompt variations, measure their business and operational impact, and iterate to enhance accuracy, consistency, and safety Regression Testing & Compliance Build principled regression test suites using tools like LangFuse and Galileo to ensure prompts remain compliant and high-performing as models and use cases evolve. Collaboration Across Teams: Work closely with data science, product, legal, engineering, and operations teams to align prompt designs with business goals, operational workflows, and compliance requirements Model Adaptation & Strategy Development prompts across multiple LLMs (GPT, LLaMA, Gemini, and Checkmate's fine-tuned models), understanding model differences to optimize outputs effectively Team Leadership & Mentorship Lead a team of analysts focused on prompt evaluation and data quality analysis, guiding prioritization, experimentation, and reporting. Collaborate with ops teams for seamless deployment and feedback loops Research & Continuous Learning Stay up to date on emerging prompting techniques, LLM behaviors, evaluation frameworks, and AI safety practices to keep Checkmate's AI solutions best-in-class
Requirements
Strong analytical and data science skills, with hands-on experience in Python (pandas, NumPy, scikit-learn) Experience designing and conducting experiments and evaluations in applied AI or NLP contexts Proficiency in SQL and working with relational databases (e.g. MySQL, PostgreSQL, Oracle, MS SQL) Good understanding of data processing, quality measurement, and testing fundamentals Experience leading analyst or operations teams, with strong prioritization, mentorship, and collaboration skills Strong problem-solving mindset with a drive to explore, optimize, and automate workflows Excellent communication skills for presenting insights to technical and non-technical stakeholders Bachelor's degree in Data Science, Computer Science, Statistics, Engineering, or a related field Flexible to work US hours until at least 6 pm ET, with a strong remote setup
Preferred Qualifications
Experience with LLM evaluation and prompt engineering workflows Familiarity with tools like LangFuse and Galileo for prompt evaluation and analysis Knowledge of cloud platforms (AWS, GCP, Azure) and data pipeline tools Familiarity with machine learning concepts and NLP workflows Master's or PhD in Data Science, Statistics, Computer Science, or a related field
Benefits
Health Care Plan (Medical, Dental & Vision) Retirement Plan (401k) Life Insurance (Basic, Voluntary & AD&D) Flexible Paid Time Off Family Leave (Maternity, Paternity) Short Term & Long Term Disability Training & Development Work From Home Stock Option Plan
Seniority level
Seniority level Mid-Senior level Employment type
Employment type Full-time Job function
Job function Other Industries IT Services and IT Consulting Referrals increase your chances of interviewing at Checkmate by 2x Sign in to set job alerts for “Prompt Engineer” roles.
We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
#J-18808-Ljbffr