Zendesk
Senior AI Agent Engineer – Voice AI
Company:
Zendesk
Job Description
The Agentic Tribe is revolutionizing the chatbot and voice assistance landscape with Gen3, a cutting‑edge AI Agent system pushing the boundaries of conversational AI. Gen3 is a goal‑oriented, dynamic, and truly conversational system capable of reasoning, planning, and adapting to user needs in real time. Leveraging a multi‑agent architecture and advanced language models, Gen3 delivers personalized and engaging user experiences, going far beyond scripted interactions to handle complex tasks and “off‑script” inquiries with ease.
We are seeking a passionate and experienced Senior Voice AI Agent Engineer with a strong focus on Voice AI to join our team. In this role you will build the cognitive architecture for our voice applications, creating systems that can reason, plan, and execute complex tasks through seamless, low‑latency spoken dialogue. You will also communicate complex technical concepts to both technical and non‑technical stakeholders.
Responsibilities
Design and develop robust, stateful, and scalable voice‑first AI agents using Python, optimized for real‑time voice interactions, managing turn‑taking, interruptions, and low‑latency responses.
Integrate best‑in‑class real‑time Speech‑to‑Text (STT), Text‑to‑Speech (TTS), and Voice Activity Detection (VAD) services to create a seamless conversational flow.
Connect voice agents with existing enterprise systems, databases, and third‑party APIs to create powerful, end‑to‑end automated workflows initiated and managed through voice.
Establish and own the evaluation framework for voice agent performance and behavior, iterating over time to systematically improve performance, reliability, and the overall user experience.
Build end‑to‑end conversational flows with reasoning, planning, and dynamic tool use—beyond pre‑scripted voice experiences.
Work cross‑functionally with product managers, ML scientists, and engineers to deeply understand user needs and voice interaction goals.
Implement fallback, recovery, and error‑handling strategies to deal with noisy audio input or speech recognition inaccuracies.
Define and track voice‑specific evaluation metrics (e.g., word error rate, latency, conversational naturalness).
Develop observability tools and guardrails to monitor performance, ensure safety, and handle edge cases in spoken interactions.
Document development, architecture decisions, and research findings to share knowledge across the team.
Requirements
Strong experience building multi‑step, tool‑using agents (LangChain, Autogen). Familiar with prompt engineering, context management, and reasoning strategies such as Chain‑of‑Thought and ReAct.
Expertise in Voice AI: building low‑latency, streaming voice applications; integrating and managing real‑time STT/TTS models and APIs; proficiency with Voice Activity Detection (VAD), noise suppression, and robust barge‑in/interruption logic; integrating third‑party voice AI APIs from providers such as OpenAI, Deepgram, ElevenLabs, etc.; understanding of latency, timing, and streaming audio constraints.
Comfortable connecting agents to external APIs, tools, and databases in secure environments.
Experience building Retrieval‑Augmented Generation pipelines with vector stores, chunking strategies, and hybrid retrieval.
Implementing and using monitoring tools and evaluation frameworks (e.g., Braintrust) to score AI agents.
Familiarity with prompt injection defense, guardrails (Rebuff, Guardrails AI), and fail‑over logic.
Token budget and latency management using caching, model routing, etc.
Expertise in Python, FastAPI, and LLM SDKs; experience deploying AI apps to cloud platforms (AWS, GCP, Azure) using CI/CD best practices.
Nice‑to‑have
M.S. or Ph.D. in Computer Science, NLP, Machine Learning, or related field.
Background in spoken dialogue systems or conversational UX design.
Familiarity with real‑time streaming architecture (e.g., WebRTC, gRPC, socket.io).
Experience with multilingual ASR/TTS pipelines.
Hybrid Work Model This role is hybrid; the team will have a rich onsite experience packed with connection, collaboration, learning, and celebration. You will have flexibility to work remotely for part of the week, but must attend the local office for part of the week. The specific in‑office schedule will be determined by the hiring manager.
About Zendesk Zendesk builds software for better customer relationships, empowering organizations to improve customer engagement and better understand their customers. Zendesk products are easy to use, implement, and allow organizations to move quickly, focus on innovation, and scale with their growth.
More than 100,000 paid customer accounts in over 150 countries and territories use Zendesk products. Zendesk has operations in the United States, Europe, Asia, Australia, and South America.
Zendesk is an equal opportunity employer, and we’re proud of our ongoing efforts to foster diversity & inclusion in the workplace. Individuals seeking employment at Zendesk are considered without regard to race, color, religion, national origin, age, sex, gender, gender identity, gender expression, sexual orientation, marital status, medical condition, ancestry, physical or mental disability, military or veteran status, or any other characteristic protected by applicable law.
By submitting your application, you agree that Zendesk may collect your personal data for recruiting, global organization planning, and related purposes. Zendesk's Candidate Privacy Notice explains what personal information Zendesk may process, where Zendesk may process your personal information, its purposes for processing your personal information, and the rights you can exercise over Zendesk’s use of your personal information.
Zendesk endeavors to make reasonable accommodations for applicants with disabilities and disabled veterans pursuant to applicable federal and state law. If you are an individual with a disability and require a reasonable accommodation to submit this application, complete any pre‑employment testing, or otherwise participate in the employee selection process, please send an e‑mail to peopleandplaces@zendesk.com with your specific accommodation request.
#J-18808-Ljbffr
Zendesk
Job Description
The Agentic Tribe is revolutionizing the chatbot and voice assistance landscape with Gen3, a cutting‑edge AI Agent system pushing the boundaries of conversational AI. Gen3 is a goal‑oriented, dynamic, and truly conversational system capable of reasoning, planning, and adapting to user needs in real time. Leveraging a multi‑agent architecture and advanced language models, Gen3 delivers personalized and engaging user experiences, going far beyond scripted interactions to handle complex tasks and “off‑script” inquiries with ease.
We are seeking a passionate and experienced Senior Voice AI Agent Engineer with a strong focus on Voice AI to join our team. In this role you will build the cognitive architecture for our voice applications, creating systems that can reason, plan, and execute complex tasks through seamless, low‑latency spoken dialogue. You will also communicate complex technical concepts to both technical and non‑technical stakeholders.
Responsibilities
Design and develop robust, stateful, and scalable voice‑first AI agents using Python, optimized for real‑time voice interactions, managing turn‑taking, interruptions, and low‑latency responses.
Integrate best‑in‑class real‑time Speech‑to‑Text (STT), Text‑to‑Speech (TTS), and Voice Activity Detection (VAD) services to create a seamless conversational flow.
Connect voice agents with existing enterprise systems, databases, and third‑party APIs to create powerful, end‑to‑end automated workflows initiated and managed through voice.
Establish and own the evaluation framework for voice agent performance and behavior, iterating over time to systematically improve performance, reliability, and the overall user experience.
Build end‑to‑end conversational flows with reasoning, planning, and dynamic tool use—beyond pre‑scripted voice experiences.
Work cross‑functionally with product managers, ML scientists, and engineers to deeply understand user needs and voice interaction goals.
Implement fallback, recovery, and error‑handling strategies to deal with noisy audio input or speech recognition inaccuracies.
Define and track voice‑specific evaluation metrics (e.g., word error rate, latency, conversational naturalness).
Develop observability tools and guardrails to monitor performance, ensure safety, and handle edge cases in spoken interactions.
Document development, architecture decisions, and research findings to share knowledge across the team.
Requirements
Strong experience building multi‑step, tool‑using agents (LangChain, Autogen). Familiar with prompt engineering, context management, and reasoning strategies such as Chain‑of‑Thought and ReAct.
Expertise in Voice AI: building low‑latency, streaming voice applications; integrating and managing real‑time STT/TTS models and APIs; proficiency with Voice Activity Detection (VAD), noise suppression, and robust barge‑in/interruption logic; integrating third‑party voice AI APIs from providers such as OpenAI, Deepgram, ElevenLabs, etc.; understanding of latency, timing, and streaming audio constraints.
Comfortable connecting agents to external APIs, tools, and databases in secure environments.
Experience building Retrieval‑Augmented Generation pipelines with vector stores, chunking strategies, and hybrid retrieval.
Implementing and using monitoring tools and evaluation frameworks (e.g., Braintrust) to score AI agents.
Familiarity with prompt injection defense, guardrails (Rebuff, Guardrails AI), and fail‑over logic.
Token budget and latency management using caching, model routing, etc.
Expertise in Python, FastAPI, and LLM SDKs; experience deploying AI apps to cloud platforms (AWS, GCP, Azure) using CI/CD best practices.
Nice‑to‑have
M.S. or Ph.D. in Computer Science, NLP, Machine Learning, or related field.
Background in spoken dialogue systems or conversational UX design.
Familiarity with real‑time streaming architecture (e.g., WebRTC, gRPC, socket.io).
Experience with multilingual ASR/TTS pipelines.
Hybrid Work Model This role is hybrid; the team will have a rich onsite experience packed with connection, collaboration, learning, and celebration. You will have flexibility to work remotely for part of the week, but must attend the local office for part of the week. The specific in‑office schedule will be determined by the hiring manager.
About Zendesk Zendesk builds software for better customer relationships, empowering organizations to improve customer engagement and better understand their customers. Zendesk products are easy to use, implement, and allow organizations to move quickly, focus on innovation, and scale with their growth.
More than 100,000 paid customer accounts in over 150 countries and territories use Zendesk products. Zendesk has operations in the United States, Europe, Asia, Australia, and South America.
Zendesk is an equal opportunity employer, and we’re proud of our ongoing efforts to foster diversity & inclusion in the workplace. Individuals seeking employment at Zendesk are considered without regard to race, color, religion, national origin, age, sex, gender, gender identity, gender expression, sexual orientation, marital status, medical condition, ancestry, physical or mental disability, military or veteran status, or any other characteristic protected by applicable law.
By submitting your application, you agree that Zendesk may collect your personal data for recruiting, global organization planning, and related purposes. Zendesk's Candidate Privacy Notice explains what personal information Zendesk may process, where Zendesk may process your personal information, its purposes for processing your personal information, and the rights you can exercise over Zendesk’s use of your personal information.
Zendesk endeavors to make reasonable accommodations for applicants with disabilities and disabled veterans pursuant to applicable federal and state law. If you are an individual with a disability and require a reasonable accommodation to submit this application, complete any pre‑employment testing, or otherwise participate in the employee selection process, please send an e‑mail to peopleandplaces@zendesk.com with your specific accommodation request.
#J-18808-Ljbffr