AssemblyAI
About AssemblyAI
At AssemblyAI, we’re building at the forefront of Speech AI, creating powerful models for speech‑to‑text and speech understanding available through a straightforward API. With more than 200,000 developers building on our API and over 5,000 paying customers, AssemblyAI is helping unlock and support the next generation of powerful, meaningful products built with AI.
Job Overview We’re seeking an exceptional Senior Software Engineer to join our LLM team. This role is focused on building and maintaining our LLM gateway service—a unified API platform that connects customers to multiple LLM providers. You’ll work on high‑impact projects that directly solve customer problems, improve their AI and agentic workflows, and ensure reliable access to the best models for their use cases.
This is a deeply customer‑focused role. You’ll work closely with our customer success team to understand customer challenges, help optimize their prompt strategies, build features that address their pain points, and ensure our service reliably delivers value. As a Senior Engineer, you’ll take ownership of significant features and integrations while mentoring more junior engineers. You should be passionate about writing clean, maintainable code, implementing comprehensive testing strategies, and building highly reliable systems in service of real customer problems.
Key Responsibilities Solve Customer Problems
Partner closely with the customer success team to understand customer use cases, challenges, and integration needs
Translate customer pain points surfaced by the CS team into technical solutions and product improvements
Build features and tooling that directly address customer needs and improve their workflows
Provide technical guidance and expertise to the customer success team to help them support customers effectively
Drive Technical Execution
Own and deliver complete features and integrations within our LLM gateway service
Build and maintain integrations with multiple LLM providers and AI services (OpenAI, Anthropic, Google Vertex, AWS Bedrock, etc.)
Write clean, maintainable, well‑tested code following best practices
Design and implement scalable, fault‑tolerant solutions with appropriate abstractions
Proactively identify and address technical debt, reliability issues, and code quality concerns
Participate in on‑call rotation to ensure service reliability and rapid incident response
Elevate Engineering Standards
Conduct thorough code reviews focused on maintainability, testing, reliability, and architectural concerns
Ensure proper test coverage across unit, integration, and end‑to‑end testing levels
Improve code maintainability and extensibility through targeted refactoring
Contribute to runbooks, incident postmortems, customer‑facing documentation, and knowledge sharing
Identify bottlenecks and propose solutions to improve team velocity and system reliability
Build and Collaborate
Work closely with product, customer success, and engineering teams to deliver solutions
Collaborate with external API providers to troubleshoot integration issues and improve reliability
Translate ambiguous customer requirements into well‑scoped technical work
Proactively communicate status, risks, and blockers
Mentor junior engineers on coding standards, reliability practices, and customer‑centric thinking
Participate in architectural discussions and contribute to technical decisions
What You'll Need Core Requirements
5+ years of professional software engineering experience
Strong proficiency in Go with demonstrated ability to write production‑quality code
Hands‑on experience with LLM APIs (OpenAI, Anthropic, etc.) and understanding of prompt engineering concepts
Experience building or working with API gateways, proxies, and rate limiting systems
Deep experience with prompt engineering, chain‑of‑thought, few‑shot learning, and other LLM optimization techniques
Experience building applications with llms or ai agents with cascading architectures
Solid understanding of software engineering fundamentals:
Data structures and algorithms
System design and architectural patterns
Testing strategies (unit, integration, end‑to‑end)
Code review practices and technical collaboration
Experience with:
RESTful APIs and distributed systems concepts
Building or integrating with third‑party APIs
SQL and database design
Version control (Git) and CI/CD pipelines
Containerization (Docker) and cloud infrastructure
Strong communication skills and ability to collaborate effectively with cross‑functional teams
Track record of delivering high‑quality, reliable software in a fast‑paced, team environment
Ability to thrive in a startup environment with changing priorities and rapid iteration
Nice to Haves
Experience with AWS services (ECS, Lambda, ElastiCache/Redis, RDS, S3, CloudWatch, API Gateway)
Experience with monitoring and observability tools (Datadog, CloudWatch)
Experience with request routing, load balancing, and traffic management
Experience with circuit breakers, retry logic, fallback strategies, and fault tolerance patterns
Understanding of API design, versioning, and backward compatibility
Experience working closely with customer success or support teams
What We're Looking For
Customer‑obsessed
– You genuinely care about solving customer problems and making their lives easier, working through the customer success team to understand and address their needs
Excellent at software fundamentals
– You write code that others want to emulate
Quality‑focused
– You care deeply about testing, documentation, and maintainability
Reliability‑obsessed
– You understand that uptime and performance directly impact customer trust
LLM‑savvy
– You understand prompt engineering, model capabilities, and how to help customers get the best results
Integration‑savvy
– You’ve wrestled with third‑party APIs and know how to build resilient integrations
Collaborative
– You work well with diverse stakeholders and partner effectively with customer success teams
Communicative
– You can explain complex technical concepts clearly to teammates and cross‑functional partners
Growth‑minded
– You’re curious, eager to learn, and want to expand into platform and infrastructure engineering
Pragmatic
– You balance perfection with delivery and understand trade‑offs
Team‑oriented
– You improve not just the code, but the team’s overall effectiveness
Proactive
– You anticipate problems before they become incidents and build systems defensively
Empathetic
– You put yourself in the customer’s shoes and design solutions accordingly
Compensation and Benefits Pay Transparency:
AssemblyAI strives to recruit and retain exceptional talent from diverse backgrounds while ensuring pay equity across our team. Our salary ranges are set to be competitive for our size, stage, and industry, and reflect just one component of the full compensation, benefits, and rewards we offer.
The range provided applies to candidates located in the
United States . For candidates outside of the U.S., compensation ranges may differ; any adjustments will be communicated throughout the interview process.
Salary range: $180,000 - $240,000
The expected base compensation for this role is listed above. Our total compensation package includes competitive equity grants, 100% employer‑paid benefits, and the flexibility of being fully remote.
Working at AssemblyAI We are a small but mighty group of startup veterans and experienced AI researchers with over 20 years of expertise in Machine Learning, Speech Recognition, and NLP. As a fully remote team, we’re looking for people to join our team who are ambitious, curious, and lead with integrity. We’re still in the early days of AI and of AssemblyAI’s journey, and are looking for teammates who won’t just fit in, but will help us define and build our company culture.
We’re committed to creating a space where our employees can bring their full selves to work and have equal opportunity to succeed. No matter your race, gender identity or expression, sexual orientation, religion, origin, ability, age, veteran status, if joining this mission speaks to you, we encourage you to apply!
Job Details Seniority Level:
Mid‑Senior level Employment Type:
Full‑time Job Function:
Engineering and Information Technology Industries:
Software Development
#J-18808-Ljbffr
Job Overview We’re seeking an exceptional Senior Software Engineer to join our LLM team. This role is focused on building and maintaining our LLM gateway service—a unified API platform that connects customers to multiple LLM providers. You’ll work on high‑impact projects that directly solve customer problems, improve their AI and agentic workflows, and ensure reliable access to the best models for their use cases.
This is a deeply customer‑focused role. You’ll work closely with our customer success team to understand customer challenges, help optimize their prompt strategies, build features that address their pain points, and ensure our service reliably delivers value. As a Senior Engineer, you’ll take ownership of significant features and integrations while mentoring more junior engineers. You should be passionate about writing clean, maintainable code, implementing comprehensive testing strategies, and building highly reliable systems in service of real customer problems.
Key Responsibilities Solve Customer Problems
Partner closely with the customer success team to understand customer use cases, challenges, and integration needs
Translate customer pain points surfaced by the CS team into technical solutions and product improvements
Build features and tooling that directly address customer needs and improve their workflows
Provide technical guidance and expertise to the customer success team to help them support customers effectively
Drive Technical Execution
Own and deliver complete features and integrations within our LLM gateway service
Build and maintain integrations with multiple LLM providers and AI services (OpenAI, Anthropic, Google Vertex, AWS Bedrock, etc.)
Write clean, maintainable, well‑tested code following best practices
Design and implement scalable, fault‑tolerant solutions with appropriate abstractions
Proactively identify and address technical debt, reliability issues, and code quality concerns
Participate in on‑call rotation to ensure service reliability and rapid incident response
Elevate Engineering Standards
Conduct thorough code reviews focused on maintainability, testing, reliability, and architectural concerns
Ensure proper test coverage across unit, integration, and end‑to‑end testing levels
Improve code maintainability and extensibility through targeted refactoring
Contribute to runbooks, incident postmortems, customer‑facing documentation, and knowledge sharing
Identify bottlenecks and propose solutions to improve team velocity and system reliability
Build and Collaborate
Work closely with product, customer success, and engineering teams to deliver solutions
Collaborate with external API providers to troubleshoot integration issues and improve reliability
Translate ambiguous customer requirements into well‑scoped technical work
Proactively communicate status, risks, and blockers
Mentor junior engineers on coding standards, reliability practices, and customer‑centric thinking
Participate in architectural discussions and contribute to technical decisions
What You'll Need Core Requirements
5+ years of professional software engineering experience
Strong proficiency in Go with demonstrated ability to write production‑quality code
Hands‑on experience with LLM APIs (OpenAI, Anthropic, etc.) and understanding of prompt engineering concepts
Experience building or working with API gateways, proxies, and rate limiting systems
Deep experience with prompt engineering, chain‑of‑thought, few‑shot learning, and other LLM optimization techniques
Experience building applications with llms or ai agents with cascading architectures
Solid understanding of software engineering fundamentals:
Data structures and algorithms
System design and architectural patterns
Testing strategies (unit, integration, end‑to‑end)
Code review practices and technical collaboration
Experience with:
RESTful APIs and distributed systems concepts
Building or integrating with third‑party APIs
SQL and database design
Version control (Git) and CI/CD pipelines
Containerization (Docker) and cloud infrastructure
Strong communication skills and ability to collaborate effectively with cross‑functional teams
Track record of delivering high‑quality, reliable software in a fast‑paced, team environment
Ability to thrive in a startup environment with changing priorities and rapid iteration
Nice to Haves
Experience with AWS services (ECS, Lambda, ElastiCache/Redis, RDS, S3, CloudWatch, API Gateway)
Experience with monitoring and observability tools (Datadog, CloudWatch)
Experience with request routing, load balancing, and traffic management
Experience with circuit breakers, retry logic, fallback strategies, and fault tolerance patterns
Understanding of API design, versioning, and backward compatibility
Experience working closely with customer success or support teams
What We're Looking For
Customer‑obsessed
– You genuinely care about solving customer problems and making their lives easier, working through the customer success team to understand and address their needs
Excellent at software fundamentals
– You write code that others want to emulate
Quality‑focused
– You care deeply about testing, documentation, and maintainability
Reliability‑obsessed
– You understand that uptime and performance directly impact customer trust
LLM‑savvy
– You understand prompt engineering, model capabilities, and how to help customers get the best results
Integration‑savvy
– You’ve wrestled with third‑party APIs and know how to build resilient integrations
Collaborative
– You work well with diverse stakeholders and partner effectively with customer success teams
Communicative
– You can explain complex technical concepts clearly to teammates and cross‑functional partners
Growth‑minded
– You’re curious, eager to learn, and want to expand into platform and infrastructure engineering
Pragmatic
– You balance perfection with delivery and understand trade‑offs
Team‑oriented
– You improve not just the code, but the team’s overall effectiveness
Proactive
– You anticipate problems before they become incidents and build systems defensively
Empathetic
– You put yourself in the customer’s shoes and design solutions accordingly
Compensation and Benefits Pay Transparency:
AssemblyAI strives to recruit and retain exceptional talent from diverse backgrounds while ensuring pay equity across our team. Our salary ranges are set to be competitive for our size, stage, and industry, and reflect just one component of the full compensation, benefits, and rewards we offer.
The range provided applies to candidates located in the
United States . For candidates outside of the U.S., compensation ranges may differ; any adjustments will be communicated throughout the interview process.
Salary range: $180,000 - $240,000
The expected base compensation for this role is listed above. Our total compensation package includes competitive equity grants, 100% employer‑paid benefits, and the flexibility of being fully remote.
Working at AssemblyAI We are a small but mighty group of startup veterans and experienced AI researchers with over 20 years of expertise in Machine Learning, Speech Recognition, and NLP. As a fully remote team, we’re looking for people to join our team who are ambitious, curious, and lead with integrity. We’re still in the early days of AI and of AssemblyAI’s journey, and are looking for teammates who won’t just fit in, but will help us define and build our company culture.
We’re committed to creating a space where our employees can bring their full selves to work and have equal opportunity to succeed. No matter your race, gender identity or expression, sexual orientation, religion, origin, ability, age, veteran status, if joining this mission speaks to you, we encourage you to apply!
Job Details Seniority Level:
Mid‑Senior level Employment Type:
Full‑time Job Function:
Engineering and Information Technology Industries:
Software Development
#J-18808-Ljbffr