Essential Software Inc.
Senior QA Engineer (AI Systems & Automation)
Essential Software Inc., Rockville, Maryland, us, 20849
Senior QA Engineer (AI Systems & Automation)
Essential Software Inc. (ESI)
Overview Essential Software Inc. is a trusted partner to federal agencies, including the National Cancer Institute (NCI), delivering secure, cloud-based platforms that support large-scale cancer data and biomedical research.
As a Senior QA Engineer (AI Systems & Automation), you will lead quality strategy and test automation for critical data platforms and AI-powered experiences. You will ensure both traditional software and AI/agentic systems are reliable, explainable, and safe in a federal, mission-driven environment.
You will:
Own end-to-end quality for complex web, API, data, and AI/ML-powered features
Design AI-aware test strategies and automation that leverage GenAI and agentic frameworks
Mentor QA engineers and collaborate closely with cross-functional teams and government partners
Roles and Responsibilities Core QA & Automation
Develop and maintain test plans, test cases, traceability, and test data for product and AI features
Execute manual and automated tests for web applications, RESTful APIs, data workflows, and AI/ML features
Own automated regression suites, release readiness criteria, and provide clear go / no-go quality signals
Participate in agile ceremonies, validate end-to-end functionality, and ensure user stories (including AI features) meet acceptance criteria
Manage the full defect lifecycle, including triage, prioritization, root cause analysis, and verification of fixes
Maintain QA documentation, runbooks, and quality dashboards
AI & Agentic QA Systems
Design and execute test strategies for AI/LLM-powered capabilities, including virtual agents, chatbots, copilots, and RAG-based systems
Use LLM-powered tools (e.g., ChatGPT, Claude, Copilot) to accelerate test design, data generation, exploratory testing, and script authoring
Build and refine QA-focused AI agents that can:
Scrape UI and verify DOM structures
Validate data against backend or ground-truth sources
Auto-generate and maintain test scripts
Run self-correcting / autonomous test flows
Evaluate and integrate agentic frameworks (e.g., OpenAI Assistants API, AWS Bedrock Agents, LangGraph, MCP) into QA workflows
Define and monitor AI-specific quality metrics (accuracy vs. ground truth, hallucination and error rates, safety / policy adherence)
Ensure AI and virtual agent experiences are accurate, consistent, and high quality in a federal context
Non-Functional, Data Quality & Collaboration
Plan and execute performance, load, and scalability testing (e.g., JMeter or equivalent)
Validate data integrity and transformation quality across complex biomedical data pipelines and AI-enhanced workflows
Partner with engineers and data scientists to ensure AI/ML models and integrations are testable, observable, and measurable post-deployment
Mentor QA team members in both traditional and AI-augmented QA practices
Collaborate with development, DevOps, product, UX, and data teams to improve testability, shift-left quality, and increase automated coverage
Integrate automation into CI/CD (e.g., GitHub Actions, Jenkins, Azure DevOps, GitLab CI), monitor test health and flakiness, and address coverage gaps
Communicate quality risks, trends, and mitigation plans to technical and non-technical stakeholders, including government partners
Basic Qualifications
Bachelor’s degree in computer science, Information Technology, Engineering, or related field
5+ years of software QA experience (manual and automation) in production environments
2+ years providing technical or process leadership (e.g., lead QA, primary product QA owner, mentor, or manager)
Strong experience with UI automation tools (Selenium WebDriver, Playwright, or Cypress)
Experience testing RESTful APIs and microservices architectures
Hands‑on experience integrating automated tests into CI/CD pipelines (GitHub Actions, Jenkins, Azure DevOps, or GitLab CI)
Professional proficiency in Python or JavaScript for test automation
Hands‑on use of GenAI tools (e.g., ChatGPT, Claude, Copilot) for QA tasks such as test‑case generation, data creation, and exploratory testing
Understanding of AI/agentic concepts:
Tool‑calling / function invocation
Multi‑step / chain‑of‑thought workflows
Autonomous / self‑healing test flows
AI‑driven data comparison and validation
Experience with performance / load testing (e.g., JMeter or equivalent)
Proficiency with Jira or similar issue tracking tools
Strong written and verbal communication skills, including the ability to explain AI‑related quality risks to stakeholders
Ability to prioritize, multitask, and operate effectively in complex, mission‑driven environments
Preferred Qualifications
AWS Cloud Practitioner certification
Experience with modern automation stacks (Playwright or Cypress) and API testing tools (Postman, REST‑assured, pytest, or similar)
Experience testing AI/ML‑powered features (LLM applications, RAG systems, agents, recommendation engines, or chatbots)
Experience with one or more:
LangChain or LangGraph
AWS Bedrock Agents or OpenAI Assistants API
MCP (Multi‑Context Protocol) or similar orchestration frameworks
Experience designing or testing internal QA copilots or automation bots for test authoring or execution
Familiarity with test management tools (e.g., TestRail, Zephyr)
Knowledge of accessibility standards (WCAG) and basic security testing practices
Prior QA experience in healthcare, life sciences, biomedical informatics, or other regulated data environments
ISTQB or similar certification
How We Work
Mission driven and customer focused, working directly with scientists, product owners, and data experts
Curious and learning oriented, with encouragement to experiment with new AI tools, patterns, and technologies
Work on cutting‑edge biomedical informatics and AI projects with real‑world impact
Full‑time role; primarily remote, with approximately four days per month onsite in Rockville, MD
Competitive benefits, professional development opportunities, and a collaborative, supportive culture
Compensation Salary range for this position: $107,000 - $138,000, depending on skills and experience.
Equal Opportunity Employer Essential Software Inc. provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type, without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws. This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation, and training.
#J-18808-Ljbffr
Overview Essential Software Inc. is a trusted partner to federal agencies, including the National Cancer Institute (NCI), delivering secure, cloud-based platforms that support large-scale cancer data and biomedical research.
As a Senior QA Engineer (AI Systems & Automation), you will lead quality strategy and test automation for critical data platforms and AI-powered experiences. You will ensure both traditional software and AI/agentic systems are reliable, explainable, and safe in a federal, mission-driven environment.
You will:
Own end-to-end quality for complex web, API, data, and AI/ML-powered features
Design AI-aware test strategies and automation that leverage GenAI and agentic frameworks
Mentor QA engineers and collaborate closely with cross-functional teams and government partners
Roles and Responsibilities Core QA & Automation
Develop and maintain test plans, test cases, traceability, and test data for product and AI features
Execute manual and automated tests for web applications, RESTful APIs, data workflows, and AI/ML features
Own automated regression suites, release readiness criteria, and provide clear go / no-go quality signals
Participate in agile ceremonies, validate end-to-end functionality, and ensure user stories (including AI features) meet acceptance criteria
Manage the full defect lifecycle, including triage, prioritization, root cause analysis, and verification of fixes
Maintain QA documentation, runbooks, and quality dashboards
AI & Agentic QA Systems
Design and execute test strategies for AI/LLM-powered capabilities, including virtual agents, chatbots, copilots, and RAG-based systems
Use LLM-powered tools (e.g., ChatGPT, Claude, Copilot) to accelerate test design, data generation, exploratory testing, and script authoring
Build and refine QA-focused AI agents that can:
Scrape UI and verify DOM structures
Validate data against backend or ground-truth sources
Auto-generate and maintain test scripts
Run self-correcting / autonomous test flows
Evaluate and integrate agentic frameworks (e.g., OpenAI Assistants API, AWS Bedrock Agents, LangGraph, MCP) into QA workflows
Define and monitor AI-specific quality metrics (accuracy vs. ground truth, hallucination and error rates, safety / policy adherence)
Ensure AI and virtual agent experiences are accurate, consistent, and high quality in a federal context
Non-Functional, Data Quality & Collaboration
Plan and execute performance, load, and scalability testing (e.g., JMeter or equivalent)
Validate data integrity and transformation quality across complex biomedical data pipelines and AI-enhanced workflows
Partner with engineers and data scientists to ensure AI/ML models and integrations are testable, observable, and measurable post-deployment
Mentor QA team members in both traditional and AI-augmented QA practices
Collaborate with development, DevOps, product, UX, and data teams to improve testability, shift-left quality, and increase automated coverage
Integrate automation into CI/CD (e.g., GitHub Actions, Jenkins, Azure DevOps, GitLab CI), monitor test health and flakiness, and address coverage gaps
Communicate quality risks, trends, and mitigation plans to technical and non-technical stakeholders, including government partners
Basic Qualifications
Bachelor’s degree in computer science, Information Technology, Engineering, or related field
5+ years of software QA experience (manual and automation) in production environments
2+ years providing technical or process leadership (e.g., lead QA, primary product QA owner, mentor, or manager)
Strong experience with UI automation tools (Selenium WebDriver, Playwright, or Cypress)
Experience testing RESTful APIs and microservices architectures
Hands‑on experience integrating automated tests into CI/CD pipelines (GitHub Actions, Jenkins, Azure DevOps, or GitLab CI)
Professional proficiency in Python or JavaScript for test automation
Hands‑on use of GenAI tools (e.g., ChatGPT, Claude, Copilot) for QA tasks such as test‑case generation, data creation, and exploratory testing
Understanding of AI/agentic concepts:
Tool‑calling / function invocation
Multi‑step / chain‑of‑thought workflows
Autonomous / self‑healing test flows
AI‑driven data comparison and validation
Experience with performance / load testing (e.g., JMeter or equivalent)
Proficiency with Jira or similar issue tracking tools
Strong written and verbal communication skills, including the ability to explain AI‑related quality risks to stakeholders
Ability to prioritize, multitask, and operate effectively in complex, mission‑driven environments
Preferred Qualifications
AWS Cloud Practitioner certification
Experience with modern automation stacks (Playwright or Cypress) and API testing tools (Postman, REST‑assured, pytest, or similar)
Experience testing AI/ML‑powered features (LLM applications, RAG systems, agents, recommendation engines, or chatbots)
Experience with one or more:
LangChain or LangGraph
AWS Bedrock Agents or OpenAI Assistants API
MCP (Multi‑Context Protocol) or similar orchestration frameworks
Experience designing or testing internal QA copilots or automation bots for test authoring or execution
Familiarity with test management tools (e.g., TestRail, Zephyr)
Knowledge of accessibility standards (WCAG) and basic security testing practices
Prior QA experience in healthcare, life sciences, biomedical informatics, or other regulated data environments
ISTQB or similar certification
How We Work
Mission driven and customer focused, working directly with scientists, product owners, and data experts
Curious and learning oriented, with encouragement to experiment with new AI tools, patterns, and technologies
Work on cutting‑edge biomedical informatics and AI projects with real‑world impact
Full‑time role; primarily remote, with approximately four days per month onsite in Rockville, MD
Competitive benefits, professional development opportunities, and a collaborative, supportive culture
Compensation Salary range for this position: $107,000 - $138,000, depending on skills and experience.
Equal Opportunity Employer Essential Software Inc. provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type, without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws. This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation, and training.
#J-18808-Ljbffr