ML Systems Architect - Inference

Get AI-powered advice on this job and more exclusive features.

This range is provided by Netpreme. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.

Base pay range

$180,000.00/yr - $300,000.00/yr

Additional compensation types

Annual Bonus and Stock options

We are hiring a Machine Learning Systems Architect - Inference . You will be at the forefront of rethinking how large-scale ML inference is executed with flexible memory expansion in data centers. This role is ideal for system builders who are passionate about pushing the boundaries of performance and scale in LLM inference.

Netpreme is a Cambridge-based startup pioneering next-generation machine learning inference systems. You will join a fast-growing team of domain experts in silicon, optics, computer architecture, and computer systems to directly influence the future of ML systems.

In this role, you will

• prototype and optimize emerging ML inference systems.
• develop novel memory models for expandable vRAM.
• perform design-space exploration, implementation, and benchmarking of inference engines, both in simulations and on real hardware.

Role requirements

• MS or PhD in computer systems, ideally with a focus on LLM inference and/or distributed systems.
• familiarity with high-performance data communication systems (such as RDMA, NCCL, MPI, etc.)
• proficiency in Python, PyTorch, C/C++.
• knowledge of SOTA inference engines and their extensions (such as vLLM, SGLang, TensorRT, LMCache, etc.).
• Equity awards grant you ownership of the company, in addition to salary
• Full health, dental, vision, disability, and life insurance
• 401(k) with company matching
• 100% charity donation matching
• Flexible PTO and remote work options

Seniority level

• Seniority level

  Entry level

Employment type

• Employment type

  Full-time

Job function

• Job function

  Engineering, Information Technology, and Research

• Industries

  Software Development, Computer Hardware Manufacturing, and Data Infrastructure and Analytics

Referrals increase your chances of interviewing at Netpreme by 2x

Inferred from the description for this job

Medical insurance

Vision insurance

401(k)

Disability insurance

Get notified about new System Architect jobs in Boston, MA .

Cambridge, MA $168,000.00-$249,000.00 1 week ago

Information Technology Solutions Architect

Boston, MA $150,000.00-$175,000.00 3 weeks ago

Solutions Architect - LLJP00001792 (REMOTE)

Lexington, MA $100.00-$110.00 15 hours ago

Boston, MA $158,336.64-$193,600.00 4 days ago

Boston, MA $140,000.00-$145,000.00 5 days ago

Startup Solutions Architect, Boston District Startups

Boston, MA $118,200.00-$204,300.00 18 hours ago

IT Solutions Architect – Manhattan Active WMS

Burlington, MA $140,000.00-$222,500.00 3 weeks ago

Solution Architect (Compliance / AML)_Remote_W2

Boston, MA $118,200.00-$204,300.00 3 weeks ago

Sr. Technical Solutions Architect (Cloud Data, Life Science, ELN/LIMS) - US Remote

Boston, MA $125,250.00-$187,875.00 1 month ago

Cambridge, MA $118,603.28-$131,253.75 1 week ago

Bedford, MA $150,000.00-$185,000.00 3 weeks ago

Solutions Architect, Amazon Web Services

Boston, MA $138,200.00-$239,000.00 2 weeks ago

AI Systems & Solutions Architect, Global Services

Boston, MA $128,000.00-$192,000.00 2 weeks ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.

#J-18808-Ljbffr