Purple Drive
Preferred location is Sunnyvale, CA although will accept candidates from Bentonville, AR
Job Description:
We are seeking a skilled
Data Engineer
with expertise in
Big Data, Cloud platforms, and distributed data systems . The ideal candidate will have experience designing, building, and optimizing data pipelines, API integrations, and real-time stream-processing systems. Key Responsibilities:
Design, develop, and optimize large-scale data pipelines and ETL workflows. Work with
Java, Python, and Scala
to build scalable data solutions. Develop APIs and integrate with systems using
Node.js, GQL, and RESTful services . Implement big data solutions leveraging
Hadoop, Hive, Spark (Scala), Presto/Trino, and Data Lake architectures . Deploy and manage workflows with
Airflow, Luigi, Automic , and similar orchestration tools. Build and maintain
real-time data streaming systems
using
Storm, Spark-Streaming, and Kafka . Utilize
Vertex AI and Cloud services (AWS/Google Cloud Platform/Azure)
for analytics and ML. Ensure system reliability, scalability, and performance in distributed environments. Collaborate with cross-functional teams to deliver high-quality data solutions. Apply best practices in CI/CD, Kubernetes deployments, and monitoring. Required Skills:
Proficiency in
Java, Python, and Scala . Expertise in
big data frameworks : Hadoop, Hive, Spark (Scala). Hands-on experience with
API development
(REST, GraphQL, Node.js). Experience with
stream-processing tools : Kafka, Storm, Spark-Streaming. Knowledge of
workflow orchestration : Airflow, Luigi, Automic. Familiarity with
Presto/Trino
and distributed SQL engines. Cloud experience (AWS, GCP, Azure), with exposure to
Vertex AI . Strong understanding of
Data Lake and data warehousing concepts . Experience with Kubernetes for container orchestration. Note: Employers have access to AI tools for description enhancement. This description has been reviewed for accuracy. About Purple Drive Technologies LLC
Founded in 2007, Purple Drive is a tech solutions firm and talent partner. We help businesses navigate technology challenges and connect professionals with opportunities. We believe in transforming businesses through IT solutions and empowering technologists to grow through challenging projects and partnerships, creating success stories for clients and professionals alike. #J-18808-Ljbffr
Job Description:
We are seeking a skilled
Data Engineer
with expertise in
Big Data, Cloud platforms, and distributed data systems . The ideal candidate will have experience designing, building, and optimizing data pipelines, API integrations, and real-time stream-processing systems. Key Responsibilities:
Design, develop, and optimize large-scale data pipelines and ETL workflows. Work with
Java, Python, and Scala
to build scalable data solutions. Develop APIs and integrate with systems using
Node.js, GQL, and RESTful services . Implement big data solutions leveraging
Hadoop, Hive, Spark (Scala), Presto/Trino, and Data Lake architectures . Deploy and manage workflows with
Airflow, Luigi, Automic , and similar orchestration tools. Build and maintain
real-time data streaming systems
using
Storm, Spark-Streaming, and Kafka . Utilize
Vertex AI and Cloud services (AWS/Google Cloud Platform/Azure)
for analytics and ML. Ensure system reliability, scalability, and performance in distributed environments. Collaborate with cross-functional teams to deliver high-quality data solutions. Apply best practices in CI/CD, Kubernetes deployments, and monitoring. Required Skills:
Proficiency in
Java, Python, and Scala . Expertise in
big data frameworks : Hadoop, Hive, Spark (Scala). Hands-on experience with
API development
(REST, GraphQL, Node.js). Experience with
stream-processing tools : Kafka, Storm, Spark-Streaming. Knowledge of
workflow orchestration : Airflow, Luigi, Automic. Familiarity with
Presto/Trino
and distributed SQL engines. Cloud experience (AWS, GCP, Azure), with exposure to
Vertex AI . Strong understanding of
Data Lake and data warehousing concepts . Experience with Kubernetes for container orchestration. Note: Employers have access to AI tools for description enhancement. This description has been reviewed for accuracy. About Purple Drive Technologies LLC
Founded in 2007, Purple Drive is a tech solutions firm and talent partner. We help businesses navigate technology challenges and connect professionals with opportunities. We believe in transforming businesses through IT solutions and empowering technologists to grow through challenging projects and partnerships, creating success stories for clients and professionals alike. #J-18808-Ljbffr