NVIDIA
Overview
We are looking for a Senior Deep Learning Software Engineer to design and build our automated inference and deployment solution. As part of the team, you will be instrumental in defining a scalable architecture for DL inference with emphasis on ease-of-use and compute efficiency. Your work will span multiple layers of the DL deployment stack, encompassing developing features in high-level frameworks like PyTorch and JAX, designing and implementing a high-performance execution environment, low-level GPU optimizations and developing custom GPU kernels in CUDA and/or Triton. This is an exceptional opportunity for passionate software engineers straddling the boundaries of research and engineering, with a strong background in both machine learning fundamentals and software architecture & engineering. What Youll Be Doing Play a pivotal role in defining a modular, scalable platform to seamlessly bridge training and deployment workflowsenabling tight integration of deployment tooling with training frameworks such as Megatron and Nemo Leverage and build upon the Torch 2.0 ecosystem (TorchDynamo, torch.export, torch.compile, etc.) to analyze and extract a standardized model graph representation from arbitrary torch models for our automated deployment solution Develop support for inference optimization techniques such as speculative decoding and LoRA Collaborate with teams across NVIDIA to use performant kernel implementations within the automated deployment solution Analyze and profile GPU kernel-level performance to identify hardware and software optimization opportunities Continuously innovate on inference performance to ensure NVIDIA's inference software solutions (TRT, TRT-LLM, TRT Model Optimizer) maintain and grow their leadership
What We Need To See
Masters, PhD, or equivalent experience in Computer Science, AI, Applied Math, or related field 8+ years of relevant work or research experience in Deep Learning Excellent software design skills, including debugging, performance analysis, and test design Strong proficiency in Python, PyTorch, and related ML tools Strong algorithms and programming fundamentals Good written and verbal communication skills and the ability to work independently and collaboratively in a fast-paced environment
Ways To Stand Out From The Crowd
Contributions to PyTorch, JAX, or other Machine Learning Frameworks Knowledge of GPU architecture and compilation stack, and capability of understanding and debugging end-to-end performance Familiarity with NVIDIA's deep learning SDKs such as TensorRT Prior experience in writing high-performance GPU kernels for ML workloads in CUDA, CUTLASS, or Triton
Salary, Benefits and Other Details
Your base salary will be determined based on location, experience, and the pay of employees in similar positions. The base salary range is 184,000 USD - 287,500 USD for Level 4, and 224,000 USD - 356,500 USD for Level 5. You will also be eligible for equity and benefits. Applications for this job will be accepted at least until July 29, 2025. NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. We value diversity in our current and future employees and do not discriminate on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law. Location: Redmond, WA Evergreen note: This posting may reference related roles and updates to salary bands or requirements over time. How to Apply
Direct message the job poster from NVIDIA or apply through the official careers portal. #J-18808-Ljbffr
We are looking for a Senior Deep Learning Software Engineer to design and build our automated inference and deployment solution. As part of the team, you will be instrumental in defining a scalable architecture for DL inference with emphasis on ease-of-use and compute efficiency. Your work will span multiple layers of the DL deployment stack, encompassing developing features in high-level frameworks like PyTorch and JAX, designing and implementing a high-performance execution environment, low-level GPU optimizations and developing custom GPU kernels in CUDA and/or Triton. This is an exceptional opportunity for passionate software engineers straddling the boundaries of research and engineering, with a strong background in both machine learning fundamentals and software architecture & engineering. What Youll Be Doing Play a pivotal role in defining a modular, scalable platform to seamlessly bridge training and deployment workflowsenabling tight integration of deployment tooling with training frameworks such as Megatron and Nemo Leverage and build upon the Torch 2.0 ecosystem (TorchDynamo, torch.export, torch.compile, etc.) to analyze and extract a standardized model graph representation from arbitrary torch models for our automated deployment solution Develop support for inference optimization techniques such as speculative decoding and LoRA Collaborate with teams across NVIDIA to use performant kernel implementations within the automated deployment solution Analyze and profile GPU kernel-level performance to identify hardware and software optimization opportunities Continuously innovate on inference performance to ensure NVIDIA's inference software solutions (TRT, TRT-LLM, TRT Model Optimizer) maintain and grow their leadership
What We Need To See
Masters, PhD, or equivalent experience in Computer Science, AI, Applied Math, or related field 8+ years of relevant work or research experience in Deep Learning Excellent software design skills, including debugging, performance analysis, and test design Strong proficiency in Python, PyTorch, and related ML tools Strong algorithms and programming fundamentals Good written and verbal communication skills and the ability to work independently and collaboratively in a fast-paced environment
Ways To Stand Out From The Crowd
Contributions to PyTorch, JAX, or other Machine Learning Frameworks Knowledge of GPU architecture and compilation stack, and capability of understanding and debugging end-to-end performance Familiarity with NVIDIA's deep learning SDKs such as TensorRT Prior experience in writing high-performance GPU kernels for ML workloads in CUDA, CUTLASS, or Triton
Salary, Benefits and Other Details
Your base salary will be determined based on location, experience, and the pay of employees in similar positions. The base salary range is 184,000 USD - 287,500 USD for Level 4, and 224,000 USD - 356,500 USD for Level 5. You will also be eligible for equity and benefits. Applications for this job will be accepted at least until July 29, 2025. NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. We value diversity in our current and future employees and do not discriminate on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law. Location: Redmond, WA Evergreen note: This posting may reference related roles and updates to salary bands or requirements over time. How to Apply
Direct message the job poster from NVIDIA or apply through the official careers portal. #J-18808-Ljbffr