AI QA Tester Job at The Fountain Group in New York

Pay: $80-90/hour W2. Our company offers our consultants a suite of benefits after a qualification period including health, vision, dental, life and disability insurance.

Remote role, candidates sitting in Eastern or Central time must maintain Eastern time hours and very occasional travel may be needed.

W2 candidates only

Manager Notes

• Seeking senior QA Test Engineer profiles who can work with autonomy.
• Environment is heavy on AWS, Azure and adding GCP soon.
• Dockers and Kubernetes both desired (moving more to Kubernetes).
• Some level of AI testing of prompt engineering, hallucination, running through the gamut of tests – not required to create test cases and frameworks, as most are in place.
• Could be lightly client focused for rollouts.
• Tech Environment (not expecting all): Neo4j, Forjay, vector databases, Python, PyTorch, GraphQL, SQL/NoSQL, structured/unstructured data; Grafana/Prometheus, Kubernetes, Docker; working with all LLM providers for RAG, Grafana, Terraform, Prefect. AWS, Azure.

Original JD

• As a QA Engineer / Tester, you’ll define and drive the quality strategy for complex services, designing test approaches that cover data, services, and UI. You’ll build robust automation in Python, establish quality gates in CI/CD, and partner closely with engineers and data scientists to catch defects early and measure real-world reliability and safety. You’ll turn observability and evaluation signals into actionable improvements that accelerate safe, high-confidence releases.
• Develop and execute test plans, test cases, and test scripts to ensure software meets quality standards. Perform functional, regression, performance, and usability testing to identify and report defects. Collaborate with cross-functional teams to understand project requirements and ensure testing aligns with project objectives. Document and track defects, and work closely with developers to resolve issues. Conduct root cause analysis to identify underlying issues and suggest process improvements. Maintain test environments and test data for accurate and reliable testing. Stay up-to-date with industry trends and best practices in software testing methodologies and tools.

Qualifications

• Degree in CS/Engineering or equivalent experience.
• 5–8+ years in QA/SDET roles testing backend services, data-intensive systems, and web applications in fast-moving environments.
• Previous experience with enterprise platforms.
• Strong Python skills (pytest/unittest, fixtures, mocking); able to author maintainable test suites, utilities, and lightweight harnesses.
• Automated API testing expertise (REST/GraphQL), including schema/contract testing (OpenAPI/AsyncAPI, Pact), negative tests, and auth/SSO flows (OAuth/OIDC).
• Web UI automation experience (Playwright or Cypress), with a pragmatic approach to the test pyramid and flake-reduction techniques.
• Data and model-centric testing: data quality validation (Great Expectations/Pandera), experiment reproducibility, golden datasets, drift/robustness checks, and offline vs. online evaluation strategies.
• Familiarity with evaluation of LLM-powered features (prompt robustness, deterministic scaffolding, guardrail checks, safety/quality metrics), plus RAG/search-quality testing where applicable.
• Performance and reliability testing (load, soak, latency, scalability) using tools such as Locust, k6, or JMeter; capacity to interpret results and drive remediation with engineering.
• CI/CD quality gates (GitHub Actions/GitLab/Jenkins), test impact analysis, parallelization, and quarantine strategies for flaky tests.
• Observability-driven quality: logs/metrics/traces (OpenTelemetry, Prometheus/Grafana), synthetic monitoring, canary/shadow testing, and SLO/SLA alignment.
• Containers and orchestration (Docker/Kubernetes) and cloud fluency (AWS/Azure/GCP), including ephemeral test environments and secrets management.
• Database and messaging fundamentals (SQL, schema evolution, migrations; Kafka/Pub/Sub) and comfort validating data pipelines end-to-end.
• Security/privacy awareness across testing (PII handling, least-privilege, secrets, vulnerability scanning) and collaboration with security and compliance partners.
• Excellent communication—clear defect narratives, crisp risk/mitigation framing, and the ability to translate findings for engineers, product managers, and non-technical stakeholders.

Who We Are:

The Fountain Group is a nationwide staffing firm with over 80 Fortune 100-500 clients. Since 2001, TFG has maintained a consistent standard of excellence, and our work is broadly recognized every year through numerous industry performance awards. Our success is a team effort.

Browse our website for additional information on our company.

The Fountain Group
3407 W Martin Luther King Jr. Dr. Tampa, FL 33607

We work in Life Sciences, Clinical, Engineering, IT, and more. Above all, we specialize in people.