Purple Drive LLC
Seeking an experienced AWS Databricks Developer with 10+ years in IT, including 5+ years of hands-on experience developing and integrating cloud-based big data solutions using Databricks on AWS. The ideal candidate will have strong expertise in Java/J2EE, Python, Spark/PySpark, Scala, and SQL, capable of designing and optimizing complex data engineering pipelines in an agile environment. Familiarity with DevOps and CI/CD practices is essential to ensure scalable and reliable production workflows.
Key Responsibilities
Design, develop, and maintain scalable data engineering pipelines using Databricks on AWS cloud.
Build, optimize, and troubleshoot Spark jobs written in PySpark, Scala, and Java to handle large-scale data processing workloads.
Develop and refactor complex SQL queries for efficient data extraction and transformation.
Collaborate with data scientists, analysts, and engineers in an Agile delivery setting to meet business requirements.
Integrate data processing workflows with AWS cloud services (S3, Lambda, Glue, EMR) ensuring seamless data flow.
Implement and adhere to DevOps best practices including CI/CD pipelines using Jenkins, Git, or equivalent tools.
Perform code reviews, performance tuning, monitoring and error handling for data pipelines.
Document code, data flow diagrams, and workflows to ensure maintainability and knowledge sharing.
Drive continuous process improvements, automation, and innovation within data engineering practices.
Required Skills & Qualifications
10+ years of IT experience with at least 5 years in Databricks development on AWS.
Proficiency in Java, J2EE, Python, PySpark, Scala, and SQL programming.
Strong knowledge of Apache Spark and data engineering pipelines.
Expertise in building and optimizing complex data models and ETL/ELT processes.
Experience in agile methodologies and delivering high-quality software in fast-paced environments.
Working knowledge of CI/CD pipelines, DevOps tools such as Jenkins, Git, and infrastructure automation.
Strong problem-solving skills and excellent communication abilities.
Employers have access to artificial intelligence language tools ("AI") that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
Dice Id: 91018020
Position Id: Nans3
Posted 1 day ago
About Purple Drive Technologies LLC Founded in 2007, Purple Drive started as a tech solutions firm and has grown into a full-service consulting and talent partner. We help businesses navigate complex technology challenges while connecting top professionals with career-defining opportunities. We believe in transforming businesses through smart IT solutions and empowering technologists to grow their expertise through challenging projects and meaningful partnerships. Built on over 20 years of trusted relationships, we create success stories for both our clients and the talented professionals who drive innovation forward.
#J-18808-Ljbffr
Key Responsibilities
Design, develop, and maintain scalable data engineering pipelines using Databricks on AWS cloud.
Build, optimize, and troubleshoot Spark jobs written in PySpark, Scala, and Java to handle large-scale data processing workloads.
Develop and refactor complex SQL queries for efficient data extraction and transformation.
Collaborate with data scientists, analysts, and engineers in an Agile delivery setting to meet business requirements.
Integrate data processing workflows with AWS cloud services (S3, Lambda, Glue, EMR) ensuring seamless data flow.
Implement and adhere to DevOps best practices including CI/CD pipelines using Jenkins, Git, or equivalent tools.
Perform code reviews, performance tuning, monitoring and error handling for data pipelines.
Document code, data flow diagrams, and workflows to ensure maintainability and knowledge sharing.
Drive continuous process improvements, automation, and innovation within data engineering practices.
Required Skills & Qualifications
10+ years of IT experience with at least 5 years in Databricks development on AWS.
Proficiency in Java, J2EE, Python, PySpark, Scala, and SQL programming.
Strong knowledge of Apache Spark and data engineering pipelines.
Expertise in building and optimizing complex data models and ETL/ELT processes.
Experience in agile methodologies and delivering high-quality software in fast-paced environments.
Working knowledge of CI/CD pipelines, DevOps tools such as Jenkins, Git, and infrastructure automation.
Strong problem-solving skills and excellent communication abilities.
Employers have access to artificial intelligence language tools ("AI") that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
Dice Id: 91018020
Position Id: Nans3
Posted 1 day ago
About Purple Drive Technologies LLC Founded in 2007, Purple Drive started as a tech solutions firm and has grown into a full-service consulting and talent partner. We help businesses navigate complex technology challenges while connecting top professionals with career-defining opportunities. We believe in transforming businesses through smart IT solutions and empowering technologists to grow their expertise through challenging projects and meaningful partnerships. Built on over 20 years of trusted relationships, we create success stories for both our clients and the talented professionals who drive innovation forward.
#J-18808-Ljbffr