VLM Computer Vision Data Scientist

Get AI-powered advice on this job and more exclusive features.

Duration : - Long Term (12+ months with possible extension)

Location : - San Jose, CA (Onsite)

Must have : - Experienced in using a video-based VLM system for autonomous tasks, like industrial robots or self-driving cars or who has working experience with VLM Development & Deployment.

What is in it for you?

As a Senior Data Scientist with expertise in Vision-Language Models (VLMs) and related technologies to lead the development of efficient, cost-effective multimodal AI solutions. The ideal candidate will have experience with advanced VLM frameworks such as VILA, Isaac, and VSS, and a proven track record of implementing production-grade VLMs for training and testing in real-world environments. A background in healthcare, particularly medical devices, is highly desirable. This role will focus on exploring and deploying state-of-the-art VLM methodologies on cloud platforms like AWS or Azure.

Responsibilities: -

• VLM Development & Deployment:
• Design, train, and deploy efficient Vision-Language Models (e.g., VILA, Isaac Sim) for multimodal applications.
• Explore cost-effective methods such as knowledge distillation, modal-adaptive pruning, and LoRA fine-tuning to optimize training and inference.
• Implement scalable pipelines for training/testing VLMs on cloud platforms (AWS SageMaker, Azure ML).
• Multimodal AI Solutions:
• Develop solutions that integrate vision and language capabilities for applications like image-text matching, visual question answering (VQA), and document data extraction.
• Leverage interleaved image-text datasets and advanced techniques (e.g., cross-attention layers) to enhance model performance.
• Apply VLMs to healthcare-specific use cases such as medical imaging analysis, position detection, motion detection and measurements.
• Ensure compliance with healthcare standards while handling sensitive data.
• Evaluate trade-offs between model size, performance, and cost using techniques like elastic visual encoders or lightweight architectures.
• Benchmark different VLMs (e.g., GPT-4V, Claude 3.5) for accuracy, speed, and cost-effectiveness on specific tasks.
• Collaborate with cross-functional teams including engineers and domain experts to define project requirements.
• Mentor junior team members and provide technical leadership on complex projects.

Experience: -

• 10+ Years

Educational Qualifications: -

• Education: Master’s or Ph.D. in Computer Science, Data Science, Machine Learning, or a related field.

Mandatory skills

• Minimum of 10+ years of experience in machine learning or data science roles with a focus on vision-language models.
• Proven expertise in deploying production-grade multimodal AI solutions.
• Experience in healthcare or medical devices is highly preferred.
• Technical Skills :
• Proficiency in Python and ML frameworks (e.g., PyTorch, TensorFlow).
• Hands-on experience with VLMs such as VILA, Isaac Sim, or VSS.
• Familiarity with cloud platforms like AWS SageMaker or Azure ML Studio for scalable AI deployment.
• Domain Knowledge:
• Understanding of medical datasets (e.g., imaging data) and healthcare regulations.
• Soft Skills:
• Strong problem-solving skills with the ability to optimize models for real-world constraints.
• Excellent communication skills to explain technical concepts to diverse stakeholders

Good to have skills: -

• Multimodal Techniques: Cross-attention layers, interleaved image-text datasets
• MLOps Tools: Docker, MLflow

Recruiter Details : -

Email I'd : -

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Contract

Job function

• Job function

  Information Technology

• Industries

  IT Services and IT Consulting

Referrals increase your chances of interviewing at VBeyond Corporation by 2x

Get notified about new Computer Scientist jobs in San Jose, CA .

Mountain View, CA $166,000.00-$244,000.00 1 week ago

Senior Manager, Data Science – Conversational AI

Manager, Developer Technology - Emerging Workloads

Mountain View, CA $204,000.00-$259,000.00 1 week ago

Sr. Software Engineer, HIL Automation, Autonomy

Senior Software Development Engineer, Virtual Network

San Jose, CA $194,000.00-$410,000.00 2 weeks ago

Senior Software Engineer, Fabric Networking - GPU

Santa Clara, CA $148,000.00-$287,500.00 4 days ago

Sr. Software Engineer, Supply Chain Applications

Sr. Software Engineer, Plant Modeling and Tools

Senior Backend Software Engineer, TikTok Customer Service Platform

San Jose, CA $194,000.00-$355,000.00 3 weeks ago

Sr Principal Engineer Software (AIOps for NGFW)

Senior Software Engineer, Audio-Video Processing (Req ID: 2025-15)

Palo Alto, CA $154,000.00-$237,150.00 3 weeks ago

Senior Software Engineer, ASIC Verification Tools

Sunnyvale, CA $155,000.00-$215,000.00 5 days ago

Redwood City, CA $140,000.00-$198,000.00 1 week ago

Sr. SW Engineering Technical Lead (Kernel Development)

San Jose, CA $198,600.00-$282,900.00 3 weeks ago

Senior Hardware Modeling Simulation SDE, AWS Machine Learning Accelerators

Senior TPM - ML Science, Deep Science for Systems and Services

Sunnyvale, CA $151,300.00-$261,500.00 3 days ago

Senior Software Engineer, Perception, Machine Learning/Computer Vision

Mountain View, CA $183,825.00-$275,975.00 2 weeks ago

Redwood City, CA $123,000.00-$185,000.00 1 week ago

Mountain View, CA $165,500.00-$223,500.00 2 hours ago

Mountain View, CA $117,000.00-$192,000.00 3 weeks ago

Santa Clara, CA $155,000.00-$300,000.00 3 days ago

(Future Opportunities) Senior Software Engineer, C++

San Francisco Bay Area $120,000.00-$155,000.00 1 month ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.