Andromeda
Staff Perception Engineer (US Based)
Andromeda, San Francisco, California, United States, 94199
Base Pay Range
$150,000.00/yr - $246,000.00/yr
Location: San Francisco, CA.
Position: Full-time (5 days in the office).
About Us Andromeda Robotics is an ambitious social robotics company with offices in Melbourne and San Francisco, dedicated to creating robots that seamlessly and intelligently interact with the human world. Our first robot, Abi, is a custom‑built platform designed from the ground up to be a helpful aid and intuitive partner in aged care homes. We are a passionate, collaborative team of engineers who solve some of the most challenging problems in AI and robotics.
Our Values
Deeply empathetic – Kindness and compassion are at the heart of everything we do
Purposely playful – Play sharpens focus. It keeps us curious, fast and obsessed with the craft
Relentlessly striving – With relentless ambition, an action bias and constant curiosity, we don’t settle
Strong when it counts – Tenacious under pressure, we expect problems and stay in motion to adapt and progress
United in action – Different minds. Shared mission. No passengers
The Role This role balances architectural leadership with strategic, hands‑on work. You will lead the design and implementation of Abi’s perception stack, translating raw sensor data into semantic understanding that supports autonomous navigation and conversational AI. This dual challenge demands a systems thinker who can architect scalable perception pipelines and also implement foundational components to prove the architecture’s effectiveness. You will bridge multiple domains—sensor fusion, computer vision, audio processing, and ML deployment—while optimising for real‑time performance on our embedded Jetson Orin AGX platform.
The Team You’ll be the founding perception hire, collaborating deeply with autonomy, conversational AI, gestures, controls, audio engineering, and ML teams. You’ll work with product owners to translate user needs into technical requirements, and with the broader engineering team to ensure your perception outputs enable downstream systems to flourish.
Key Responsibilities
Architect the Perception Stack: Design and own the full system from raw sensors through semantic understanding, including interface contracts, compute and latency budgeting, and robust architectural decisions
Lead Cross‑Functional Design: Facilitate design workshops with autonomy, conversational AI, audio engineering, ML and hardware teams to align resources and interface definitions
Implement Strategically: Develop core perception capabilities such as face recognition, speaker diarisation, person detection and tracking, and sensor fusion to validate and instantiate the architecture
Own Production Systems: Build with production‑readiness in mind, including graceful degradation, monitoring, debugging tools and deployment pipelines
Make Technical Decisions: Drive build vs. buy, algorithm choices and embedded deployment optimisations, focusing on performance and resource constraints
Scale the Team: Support recruitment and mentoring as the perception team grows
What You’ll Build
Scene Awareness Pipeline: Architect the unified perception system that serves navigation, gesture and conversational needs
Person Detection & Tracking: Implement robust multi‑person tracking using multiple cameras, LiDAR point clouds and IMU data for both navigation obstacle avoidance and social scene understanding
Audio‑Visual Speaker Localisation: Fuse microphone arrays, face detection, gaze tracking and skeleton pose to determine who is speaking and where Abi should direct her attention
Sensor Fusion Architecture: Design the common data structures and synchronisation primitives that enable downstream systems to consume perception outputs efficiently, optimised to run on Jetson Orin AGX within strict latency budgets
Your First 90 Days
Month 1:
Architect the Perception Stack, lead design workshops, establish interface contracts, allocate compute and latency budgets, define data schemas, synchronisation primitives and failure mode handling. Deliver the architectural blueprint that guides all perception work.
Month 2:
Validate with Face Recognition & Speaker Diarisation. Implement face recognition and speaker diarisation systems to prove out the architecture. Use these as forcing functions to stress‑test your interface definitions, budget allocations and integration patterns. Iterate the architecture based on real deployment constraints.
Month 3:
Establish Production Infrastructure. Build monitoring, testing frameworks and deployment pipelines. Codify the architectural patterns and interface contracts. Prepare the foundation for the team to scale perception capabilities independently.
Requirements
Bachelor’s or Master’s degree in Computer Science, Robotics, Electrical Engineering or related field
5‑7+ years of experience building and shipping perception systems for robotics or autonomous vehicles
Deep expertise in computer vision (object detection, tracking, 3D reconstruction, camera calibration)
Strong fundamentals in sensor fusion (Kalman filtering, probabilistic estimation, multi‑modal integration)
Real‑time embedded systems experience (CUDA, TensorRT, ROS2)
Proven architectural skills with hands‑on coding ability in C++ and Python
Pragmatic mindset balancing off‑the‑shelf and custom solutions
Bonus Points If You Have
Audio processing background, including beamforming and source localisation
Experience with face recognition systems and liveness detection
Expertise with depth sensors such as stereo vision, structured light or LiDAR
Skills in human pose estimation and social gaze prediction mechanisms
Experience optimising ML models for edge deployment
A PhD in relevant fields (computer vision, robotics, signal processing)
Interview Process
1st Round: Recruiter phone screen to discuss background and fit
Take‑home assessment to architect the system and perform coding tasks involving datasets to fuse audio and visual inputs
2nd Round: Technical interview focusing on perception engineering concepts, system design and implementation challenges
Final Round: Cultural interview to ensure alignment with Andromeda Robotics’ values and team fit
Benefits The expected base salary range for this role, when performed in our San Francisco office, is $150,000 – $250,000 USD, depending on skills and experience. The total compensation package may also include additional benefits or components based on the specific role. Details will be provided if an employment offer is made.
We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, colour, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.
Note:
At this time, we are generally not offering visa sponsorship for this role.
#J-18808-Ljbffr
Location: San Francisco, CA.
Position: Full-time (5 days in the office).
About Us Andromeda Robotics is an ambitious social robotics company with offices in Melbourne and San Francisco, dedicated to creating robots that seamlessly and intelligently interact with the human world. Our first robot, Abi, is a custom‑built platform designed from the ground up to be a helpful aid and intuitive partner in aged care homes. We are a passionate, collaborative team of engineers who solve some of the most challenging problems in AI and robotics.
Our Values
Deeply empathetic – Kindness and compassion are at the heart of everything we do
Purposely playful – Play sharpens focus. It keeps us curious, fast and obsessed with the craft
Relentlessly striving – With relentless ambition, an action bias and constant curiosity, we don’t settle
Strong when it counts – Tenacious under pressure, we expect problems and stay in motion to adapt and progress
United in action – Different minds. Shared mission. No passengers
The Role This role balances architectural leadership with strategic, hands‑on work. You will lead the design and implementation of Abi’s perception stack, translating raw sensor data into semantic understanding that supports autonomous navigation and conversational AI. This dual challenge demands a systems thinker who can architect scalable perception pipelines and also implement foundational components to prove the architecture’s effectiveness. You will bridge multiple domains—sensor fusion, computer vision, audio processing, and ML deployment—while optimising for real‑time performance on our embedded Jetson Orin AGX platform.
The Team You’ll be the founding perception hire, collaborating deeply with autonomy, conversational AI, gestures, controls, audio engineering, and ML teams. You’ll work with product owners to translate user needs into technical requirements, and with the broader engineering team to ensure your perception outputs enable downstream systems to flourish.
Key Responsibilities
Architect the Perception Stack: Design and own the full system from raw sensors through semantic understanding, including interface contracts, compute and latency budgeting, and robust architectural decisions
Lead Cross‑Functional Design: Facilitate design workshops with autonomy, conversational AI, audio engineering, ML and hardware teams to align resources and interface definitions
Implement Strategically: Develop core perception capabilities such as face recognition, speaker diarisation, person detection and tracking, and sensor fusion to validate and instantiate the architecture
Own Production Systems: Build with production‑readiness in mind, including graceful degradation, monitoring, debugging tools and deployment pipelines
Make Technical Decisions: Drive build vs. buy, algorithm choices and embedded deployment optimisations, focusing on performance and resource constraints
Scale the Team: Support recruitment and mentoring as the perception team grows
What You’ll Build
Scene Awareness Pipeline: Architect the unified perception system that serves navigation, gesture and conversational needs
Person Detection & Tracking: Implement robust multi‑person tracking using multiple cameras, LiDAR point clouds and IMU data for both navigation obstacle avoidance and social scene understanding
Audio‑Visual Speaker Localisation: Fuse microphone arrays, face detection, gaze tracking and skeleton pose to determine who is speaking and where Abi should direct her attention
Sensor Fusion Architecture: Design the common data structures and synchronisation primitives that enable downstream systems to consume perception outputs efficiently, optimised to run on Jetson Orin AGX within strict latency budgets
Your First 90 Days
Month 1:
Architect the Perception Stack, lead design workshops, establish interface contracts, allocate compute and latency budgets, define data schemas, synchronisation primitives and failure mode handling. Deliver the architectural blueprint that guides all perception work.
Month 2:
Validate with Face Recognition & Speaker Diarisation. Implement face recognition and speaker diarisation systems to prove out the architecture. Use these as forcing functions to stress‑test your interface definitions, budget allocations and integration patterns. Iterate the architecture based on real deployment constraints.
Month 3:
Establish Production Infrastructure. Build monitoring, testing frameworks and deployment pipelines. Codify the architectural patterns and interface contracts. Prepare the foundation for the team to scale perception capabilities independently.
Requirements
Bachelor’s or Master’s degree in Computer Science, Robotics, Electrical Engineering or related field
5‑7+ years of experience building and shipping perception systems for robotics or autonomous vehicles
Deep expertise in computer vision (object detection, tracking, 3D reconstruction, camera calibration)
Strong fundamentals in sensor fusion (Kalman filtering, probabilistic estimation, multi‑modal integration)
Real‑time embedded systems experience (CUDA, TensorRT, ROS2)
Proven architectural skills with hands‑on coding ability in C++ and Python
Pragmatic mindset balancing off‑the‑shelf and custom solutions
Bonus Points If You Have
Audio processing background, including beamforming and source localisation
Experience with face recognition systems and liveness detection
Expertise with depth sensors such as stereo vision, structured light or LiDAR
Skills in human pose estimation and social gaze prediction mechanisms
Experience optimising ML models for edge deployment
A PhD in relevant fields (computer vision, robotics, signal processing)
Interview Process
1st Round: Recruiter phone screen to discuss background and fit
Take‑home assessment to architect the system and perform coding tasks involving datasets to fuse audio and visual inputs
2nd Round: Technical interview focusing on perception engineering concepts, system design and implementation challenges
Final Round: Cultural interview to ensure alignment with Andromeda Robotics’ values and team fit
Benefits The expected base salary range for this role, when performed in our San Francisco office, is $150,000 – $250,000 USD, depending on skills and experience. The total compensation package may also include additional benefits or components based on the specific role. Details will be provided if an employment offer is made.
We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, colour, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.
Note:
At this time, we are generally not offering visa sponsorship for this role.
#J-18808-Ljbffr