Join to apply for the Founding Backend Engineer - LLM Orchestration role at Nestmed
3 days ago Be among the first 25 applicants
Join to apply for the Founding Backend Engineer - LLM Orchestration role at Nestmed
Get AI-powered advice on this job and more exclusive features.
This range is provided by Nestmed. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.
Base pay range
$300,000.00/yr - $360,000.00/yr
About Nestmed
Nestmed is redefining post-acute healthcare with AI-driven technology that helps clinicians work more efficiently and provide better patient care.
Within one year, we've processed over half a million patient visits, with tens of thousands of clinicians using our product daily. We're now working with 7 of the top 10 post-acute healthcare enterprises in the U.S., helping shape the future of home healthcare delivery.
Founded by Stanford and YC alumni with deep healthcare and AI expertise, our founding team combines years of clinical experience with cutting-edge technical backgrounds from companies like Amazon, Google, Meta, and leading healthcare organizations. Backed by top investors including SciFi VC (Max Levchin, PayPal co-founder) and Mischief Capital (Plaid founder), we're building the next generation of healthcare infrastructure.
About The Role
As the founding Backend Engineer on our LLM Orchestration team, you'll be deploying and managing LLMs at scale, learning how to orchestrate them in complex production scenarios that directly impact patient care. You'll rebuild and maintain our core AI inference engine that powers all of Nestmed's intelligent capabilities across several thousand clinical conversations daily.
Our system orchestrates over a dozen different AI models - both fine-tuned in-house models and third-party APIs - with low latency and high availability. You'll work on complex technical challenges like intelligent model routing based on clinical context, implementing sophisticated fallback strategies across multiple providers, optimizing inference costs through batching and caching, and ensuring clinical accuracy through comprehensive model evaluation pipelines.
This isn't about calling OpenAI APIs. You'll build sophisticated orchestration logic that selects optimal models for each clinical task, implements custom retry and circuit breaker patterns for provider failures, manages rate limits across multiple concurrent workflows, and maintains detailed performance metrics across the entire AI pipeline. You'll start as the solo engineer on this critical infrastructure and grow it into a robust team handling core AI engineering.
What You'll Do
- Build and optimize our core AI inference engine that routes requests across multiple LLM providers based on clinical context, cost optimization, and latency requirements
- Design robust model serving infrastructure with intelligent load balancing, failover mechanisms, and A/B testing frameworks for model evaluation in production
- Implement production-grade AI pipelines with comprehensive observability, distributed tracing, and real-time performance monitoring for healthcare-critical workloads
- Optimize inference costs and latency through intelligent request batching, response caching, model quantization, and dynamic provider selection algorithms
- Build custom model fine-tuning and deployment pipelines for healthcare-specific tasks using frameworks like Transformers, vLLM, and distributed training infrastructure
- Create sophisticated prompt engineering systems that dynamically optimize prompts based on clinical context and historical model performance data
- Design comprehensive evaluation frameworks that continuously monitor model accuracy, clinical safety, and regulatory compliance across all deployed models
- Build model versioning and deployment systems that support safe rollouts, instant rollbacks, and controlled experimentation in production healthcare environments
- 6+ years of backend engineering experience building high-performance distributed systems, with focus on latency-critical applications and reliability engineering
- Deep production experience with LLMs including multi-provider orchestration, custom model serving, and building reliable inference infrastructure at scale
- Strong expertise in ML infrastructure including model serving frameworks (TensorRT, vLLM, TorchServe), distributed training, and GPU optimization
- Experience with model evaluation and monitoring including A/B testing frameworks, performance monitoring, and building comprehensive observability for ML systems
- Proficiency in Python and ML frameworks with hands-on experience in model fine-tuning, prompt engineering, and deploying custom models to production
- Track record scaling ML systems with experience optimizing inference costs, managing multiple model providers, and building reliable AI infrastructure
- Understanding of healthcare or regulated industries where model accuracy, auditability, and compliance are mission-critical requirements
- San Francisco-based and excited about working closely with AI researchers to productionize cutting-edge models for healthcare applications
You'll be building the AI infrastructure that processes millions of patient interactions, directly impacting care quality for thousands of patients daily. Every optimization you make reduces healthcare costs, improves clinical accuracy, and enables new AI capabilities that transform patient outcomes.
You'll start as the founding ML infrastructure engineer and build this into a world-class AI platform team. Join us in San Francisco to build the most sophisticated LLM orchestration system in healthcare alongside leading AI researchers and clinical experts.
If you’re passionate about building high-impact products that solve real-world problems, we’d love to hear from you. Apply today!
Compensation Range: $300K - $360K
Seniority level
Seniority level
Not Applicable
Employment type
Employment type
Full-time
Job function
Job function
Engineering and Information TechnologyIndustries
Hospitals and Health Care
Referrals increase your chances of interviewing at Nestmed by 2x
Get notified about new Back End Developer jobs in San Francisco, CA .
San Francisco, CA $160,000.00-$180,000.00 2 days ago
San Francisco, CA $130,000.00-$238, hours ago
San Francisco, CA $150,000.00-$250,000.00 3 weeks ago
Full-Stack Software Engineer (Jr/Mid level)
San Francisco, CA $120,000.00-$180,000.00 1 month ago
San Francisco, CA $99,500.00-$200,000.00 2 weeks ago
San Francisco, CA $150,000.00-$230,000.00 4 months ago
San Francisco, CA $180,000.00-$280,000.00 1 day ago
Software Development Engineer I - Frontend & Mobile
San Francisco, CA $99,500.00-$200,000.00 2 weeks ago
Software Engineer Intern, Frontend - Fall 2025
San Francisco, CA $56.25-$137,000.00 3 days ago
San Francisco, CA $160,000.00-$200,000.00 2 months ago
San Francisco, CA $150,000.00-$176,000.00 3 months ago
San Francisco, CA $120,000.00-$190,000.00 9 months ago
San Francisco, CA $130,000.00-$140,000.00 2 weeks ago
Software Engineer, AI Intern (Summer 2026)
San Francisco, CA $125,000.00-$175,000.00 2 months ago
Software Engineer, AI Intern (Winter 2026)
San Francisco, CA $130,000.00-$240,000.00 2 weeks ago
San Francisco, CA $163,200.00-$223,200.00 2 days ago
Software Engineer, Frontend (All Levels)
San Francisco, CA $150,000.00-$220,000.00 2 weeks ago
San Francisco, CA $99,500.00-$200,000.00 2 weeks ago
San Francisco, CA $140,000.00-$280,000.00 8 months ago
San Francisco, CA $155,000.00-$339,500.00 2 weeks ago
San Francisco, CA $190,000.00-$250,000.00 2 weeks ago
San Francisco, CA $165,000.00-$165,000.00 2 years ago
We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
#J-18808-Ljbffr