Sully.ai
Overview
Head of Engineering, QA
at Sully.ai — you’ll build QA from the ground up, owning strategy, automation, and execution. You’ll ensure our AI medical employees are safe, reliable, and high-performing. This is a 0→1 role: you’ll be the first in function, setting standards, scaling automation, and growing a team. What You'll Do
Own QA Strategy: Define the framework for automation, functional, performance, and compliance testing. Build & Scale Automation: Lead design and implementation of Playwright-based test frameworks; automate test cases for UI and API. Ensure Reliability: Scale automation coverage, add new tests, monitor dashboards, and fix failing tests fast. Guarantee Quality: Validate accuracy, security, and compliance (HIPAA/SOC-2); partner with Engineering and Clinical teams. Lead & Grow: Recruit, mentor, and drive a high-performing QA team; embed QA into CI/CD pipelines (GitHub Actions). Must-Have
Founder / 0→1 experience — first in function, building QA processes and teams from scratch. 6–8+ years QA experience with strong Playwright expertise. 3-4+ years with strong people leadership. Proven ability to automate functional UI and API test cases. Hands-on with QA tools (Playwright, Cypress, PyTest, Postman). Experience with scaling test automation and maintaining reliability. Highly motivated, driven, and ownership-oriented. Nice to Have
Experience testing AI/LLM workflows. Dashboarding/observability tooling. GitHub Actions CI/CD integration. Prior startup or healthcare QA experience. Tech Snapshot
Automation: Playwright, Cypress, PyTest CI/CD: GitHub Actions, Kubernetes, Docker Performance: Locust, JMeter, k6 Cloud: AWS/GCP/Azure AI QA: Prompt/LLM evaluation, guardrails Why Join Sully.ai?
Build QA from 0→1:
Define how quality is done at a fast-growing AI healthcare startup. Real Impact:
Ensure clinical-grade safety and reliability in products used by doctors and patients. Remote-First:
Work with a mission-driven team from anywhere. Compensation:
Competitive salary + equity. Tackle Hard Problems:
AI QA at scale in healthcare—complex, high-impact, and career-defining. Sully.ai is an equal opportunity employer. In addition to EEO being the law, it is a policy that is fully consistent with our principles. All qualified applicants will receive consideration for employment without regard to status as a protected veteran or a qualified individual with a disability, or other protected status such as race, religion, color, national origin, sex, sexual orientation, gender identity, genetic information, pregnancy or age. Sully.ai prohibits any form of workplace harassment. Job Details
Seniority level: Director Employment type: Full-time Job function: Engineering and Information Technology Industries: Hospitals and Health Care
#J-18808-Ljbffr
Head of Engineering, QA
at Sully.ai — you’ll build QA from the ground up, owning strategy, automation, and execution. You’ll ensure our AI medical employees are safe, reliable, and high-performing. This is a 0→1 role: you’ll be the first in function, setting standards, scaling automation, and growing a team. What You'll Do
Own QA Strategy: Define the framework for automation, functional, performance, and compliance testing. Build & Scale Automation: Lead design and implementation of Playwright-based test frameworks; automate test cases for UI and API. Ensure Reliability: Scale automation coverage, add new tests, monitor dashboards, and fix failing tests fast. Guarantee Quality: Validate accuracy, security, and compliance (HIPAA/SOC-2); partner with Engineering and Clinical teams. Lead & Grow: Recruit, mentor, and drive a high-performing QA team; embed QA into CI/CD pipelines (GitHub Actions). Must-Have
Founder / 0→1 experience — first in function, building QA processes and teams from scratch. 6–8+ years QA experience with strong Playwright expertise. 3-4+ years with strong people leadership. Proven ability to automate functional UI and API test cases. Hands-on with QA tools (Playwright, Cypress, PyTest, Postman). Experience with scaling test automation and maintaining reliability. Highly motivated, driven, and ownership-oriented. Nice to Have
Experience testing AI/LLM workflows. Dashboarding/observability tooling. GitHub Actions CI/CD integration. Prior startup or healthcare QA experience. Tech Snapshot
Automation: Playwright, Cypress, PyTest CI/CD: GitHub Actions, Kubernetes, Docker Performance: Locust, JMeter, k6 Cloud: AWS/GCP/Azure AI QA: Prompt/LLM evaluation, guardrails Why Join Sully.ai?
Build QA from 0→1:
Define how quality is done at a fast-growing AI healthcare startup. Real Impact:
Ensure clinical-grade safety and reliability in products used by doctors and patients. Remote-First:
Work with a mission-driven team from anywhere. Compensation:
Competitive salary + equity. Tackle Hard Problems:
AI QA at scale in healthcare—complex, high-impact, and career-defining. Sully.ai is an equal opportunity employer. In addition to EEO being the law, it is a policy that is fully consistent with our principles. All qualified applicants will receive consideration for employment without regard to status as a protected veteran or a qualified individual with a disability, or other protected status such as race, religion, color, national origin, sex, sexual orientation, gender identity, genetic information, pregnancy or age. Sully.ai prohibits any form of workplace harassment. Job Details
Seniority level: Director Employment type: Full-time Job function: Engineering and Information Technology Industries: Hospitals and Health Care
#J-18808-Ljbffr