Advanced Hires
Overview
We are looking for a talented Data Engineer with expertise in Python data processing. The ideal candidate will have a strong background in Python API development, parallel data processing, and distributed systems design. You will be responsible for building and maintaining systems that handle large-scale data processing tasks, ensuring high performance and scalability. Responsibilities
Python API Development
Develop and maintain RESTful APIs using Python web frameworks such as FastAPI or Django. Collaborate with front-end developers to integrate user-facing elements with server-side logic.
Parallel Data Processing
Utilize Pandas, NumPy, and other libraries to process large datasets efficiently. Implement multithreading, multiprocessing, and asynchronous programming techniques. Optimize data processing pipelines to handle millions of rows with minimal latency.
Distributed Systems Design
Design and implement distributed systems with a focus on scalability and reliability. Understand and apply core concepts such as load balancing and task queues. Use Docker to containerize applications and manage dependencies. (Preferred) Experience with Kubernetes for container orchestration.
Technical Communication
Clearly articulate complex technical concepts to team members and stakeholders. Document system designs, processes, and code effectively. Collaborate with cross-functional teams to align on project goals and deliverables.
Must-Have Qualifications
Experience in Python Web Frameworks
Proficiency with FastAPI, Django, or similar frameworks. O C# coding Understanding of RESTful API principles and best practices.
Docker Knowledge
Ability to create and manage Docker Files. Experience with containerization for deployment and development workflows.
Systems Design Understanding
Basic knowledge of load balancing, task queues, and distributed system concepts. Ability to design systems that are scalable and maintainable.
Concurrent and Parallel Computing Skills
Proficiency in multithreading and multiprocessing without relying solely on external libraries or frameworks. Familiarity with asynchronous programming, particularly asyncIO in Python.
Communication Skills
Excellent technical communication abilities. Experience collaborating in team environments and conveying complex ideas clearly.
Preferred Qualifications
Education
BS or MS in Computer Science
Advanced Data Processing Tools
Experience with Polars, PySpark, or similar tools. Handling of large-scale data processing tasks efficiently.
Distributed Computing Experience
Hands-on experience with distributed architectures in Docker. Familiarity with concepts like task queuing, MapReduce, and saga patterns.
Kubernetes Experience
Knowledge of container orchestration using Kubernetes. Experience deploying and managing applications in a Kubernetes cluster.
Problem-Solving at Scale
Demonstrated ability to solve complex problems using parallel or distributed computing. Innovative thinking beyond single-threaded processes.
Compensation:
$50.00 - $55.00 per hour Location:
Onsite in Houston, TX. #J-18808-Ljbffr
We are looking for a talented Data Engineer with expertise in Python data processing. The ideal candidate will have a strong background in Python API development, parallel data processing, and distributed systems design. You will be responsible for building and maintaining systems that handle large-scale data processing tasks, ensuring high performance and scalability. Responsibilities
Python API Development
Develop and maintain RESTful APIs using Python web frameworks such as FastAPI or Django. Collaborate with front-end developers to integrate user-facing elements with server-side logic.
Parallel Data Processing
Utilize Pandas, NumPy, and other libraries to process large datasets efficiently. Implement multithreading, multiprocessing, and asynchronous programming techniques. Optimize data processing pipelines to handle millions of rows with minimal latency.
Distributed Systems Design
Design and implement distributed systems with a focus on scalability and reliability. Understand and apply core concepts such as load balancing and task queues. Use Docker to containerize applications and manage dependencies. (Preferred) Experience with Kubernetes for container orchestration.
Technical Communication
Clearly articulate complex technical concepts to team members and stakeholders. Document system designs, processes, and code effectively. Collaborate with cross-functional teams to align on project goals and deliverables.
Must-Have Qualifications
Experience in Python Web Frameworks
Proficiency with FastAPI, Django, or similar frameworks. O C# coding Understanding of RESTful API principles and best practices.
Docker Knowledge
Ability to create and manage Docker Files. Experience with containerization for deployment and development workflows.
Systems Design Understanding
Basic knowledge of load balancing, task queues, and distributed system concepts. Ability to design systems that are scalable and maintainable.
Concurrent and Parallel Computing Skills
Proficiency in multithreading and multiprocessing without relying solely on external libraries or frameworks. Familiarity with asynchronous programming, particularly asyncIO in Python.
Communication Skills
Excellent technical communication abilities. Experience collaborating in team environments and conveying complex ideas clearly.
Preferred Qualifications
Education
BS or MS in Computer Science
Advanced Data Processing Tools
Experience with Polars, PySpark, or similar tools. Handling of large-scale data processing tasks efficiently.
Distributed Computing Experience
Hands-on experience with distributed architectures in Docker. Familiarity with concepts like task queuing, MapReduce, and saga patterns.
Kubernetes Experience
Knowledge of container orchestration using Kubernetes. Experience deploying and managing applications in a Kubernetes cluster.
Problem-Solving at Scale
Demonstrated ability to solve complex problems using parallel or distributed computing. Innovative thinking beyond single-threaded processes.
Compensation:
$50.00 - $55.00 per hour Location:
Onsite in Houston, TX. #J-18808-Ljbffr