Mindlance
Summary:
We are seeking a skilled and experienced engineer to join our team as a Distributed Systems Specialist. As a generalist with expertise in distributed systems, you will play a key role in designing, developing, and optimizing our data generation pipelines for maximum efficiency. Your experience with DAGs/Graphs, large fan-out loads, and distributed workers/jobs will be instrumental in helping us scale our systems to meet the growing demands of our users.
Responsibilities: • Design, develop, and optimize data processing pipelines using DAGs/Graphs and distributed workers/jobs • Collaborate with cross-functional teams to identify and prioritize system improvements • Develop and implement solutions to handle large fan-out loads and ensure system scalability • Work closely with data scientists and engineers to integrate and scale solutions • Identify opportunities for optimization and implement changes to improve system performance and reduce resource utilization
Requirements: • 5+ years of experience in software development, with a focus on distributed systems • Experience with large fan-out loads and distributed workers/jobs • Proficiency in at least one of the following programming languages: • Python (primary language) • Rust • C++ • Hack (PHP) • Experience with cloud-based infrastructure, such as AWS or GCP • Strong problem-solving skills and ability to work independently • Excellent communication and collaboration skills
Surrounding team & key projects Focused on rendering infrastructure, the team generates large volumes of computer-generated data to support the machine learning pipeline. The necessary infrastructure for accessing compute resources is in place, enabling a seamless user experience in real-life (RL) applications.
Typical Day-to-Day in the role • SW developments in scheduling system • Authorize and submit data pipelines • Work "On call" • Collaborate with engineers who are writing pipelines
How will performance be measured? • Ability to work independently and track down problems and improve systems • Strong proven technical ability
Compelling Story & Candidate Value Proposition What makes this role interesting? This position offers the opportunity to work on systems that handle large-scale compute workloads, providing significant challenges in scalability and performance. You will take ownership and be responsible for maintaining and optimizing code written by others, fostering collaboration and shared success. The role requires a proactive mindset to seek efficiency improvements across a wide range of workloads, making a meaningful impact on system reliability and resource utilization.
Must-Have Skills 1 Experience running compute clusters or building data infrastructure 2 Proficiency in at least one of the following programming languages: Python (ideally), Rust, C++, Hack (PHP) 3 Experience with cloud-based infrastructure such as AWS or GC
Nice-to-have Skills 1 Experience with companies using large data scales
Req Intake Template 2 Experience with Digital Content Creation (DCC) tools such as blender 3 Experience with machine learning frameworks, such as TensorFlow or PyTorch
Years of overall experience required?
5+ years
Degrees/certifications required?
Bachelors with tech field related
EEO : "Mindlance is an Equal Opportunity Employer and does not discriminate in employment on the basis of - Minority/Gender/Disability/Religion/LGBTQI/Age/Veterans."
Responsibilities: • Design, develop, and optimize data processing pipelines using DAGs/Graphs and distributed workers/jobs • Collaborate with cross-functional teams to identify and prioritize system improvements • Develop and implement solutions to handle large fan-out loads and ensure system scalability • Work closely with data scientists and engineers to integrate and scale solutions • Identify opportunities for optimization and implement changes to improve system performance and reduce resource utilization
Requirements: • 5+ years of experience in software development, with a focus on distributed systems • Experience with large fan-out loads and distributed workers/jobs • Proficiency in at least one of the following programming languages: • Python (primary language) • Rust • C++ • Hack (PHP) • Experience with cloud-based infrastructure, such as AWS or GCP • Strong problem-solving skills and ability to work independently • Excellent communication and collaboration skills
Surrounding team & key projects Focused on rendering infrastructure, the team generates large volumes of computer-generated data to support the machine learning pipeline. The necessary infrastructure for accessing compute resources is in place, enabling a seamless user experience in real-life (RL) applications.
Typical Day-to-Day in the role • SW developments in scheduling system • Authorize and submit data pipelines • Work "On call" • Collaborate with engineers who are writing pipelines
How will performance be measured? • Ability to work independently and track down problems and improve systems • Strong proven technical ability
Compelling Story & Candidate Value Proposition What makes this role interesting? This position offers the opportunity to work on systems that handle large-scale compute workloads, providing significant challenges in scalability and performance. You will take ownership and be responsible for maintaining and optimizing code written by others, fostering collaboration and shared success. The role requires a proactive mindset to seek efficiency improvements across a wide range of workloads, making a meaningful impact on system reliability and resource utilization.
Must-Have Skills 1 Experience running compute clusters or building data infrastructure 2 Proficiency in at least one of the following programming languages: Python (ideally), Rust, C++, Hack (PHP) 3 Experience with cloud-based infrastructure such as AWS or GC
Nice-to-have Skills 1 Experience with companies using large data scales
Req Intake Template 2 Experience with Digital Content Creation (DCC) tools such as blender 3 Experience with machine learning frameworks, such as TensorFlow or PyTorch
Years of overall experience required?
5+ years
Degrees/certifications required?
Bachelors with tech field related
EEO : "Mindlance is an Equal Opportunity Employer and does not discriminate in employment on the basis of - Minority/Gender/Disability/Religion/LGBTQI/Age/Veterans."