Mindteck
Data Engineer (Python, Spark)
Mindteck - Columbus, Ohio, United States, 43224
Work at Mindteck
Overview
- View job
Overview
Job Responsibilities
Collaborate with the team to build out features for the data platform and consolidate data assets. Build, maintain, and optimize data pipelines using Spark. Advise, consult, and coach other data professionals on standards and practices. Work with the team to define company data assets. Migrate CMS' data platform into Chase's environment. Partner with business analysts and solutions architects to develop technical architectures for strategic enterprise projects and initiatives. Build libraries to standardize data processing methods. Demonstrate a passion for teaching and learning, understanding that continuous learning is essential for success. Have a solid understanding of AWS tools such as EMR or Glue, including their advantages and disadvantages, and be able to communicate this knowledge effectively. Implement automation in applicable processes. Mandatory Skills
5+ years of experience in a data engineering role. Proficiency in Python (or similar language) and SQL. Strong experience in building data pipelines with Spark. Excellent verbal and written communication skills. Strong analytical and problem-solving abilities. Experience with relational datastores, NoSQL datastores, and cloud object storage. Experience building data processing infrastructure in AWS. Bonus: Experience with infrastructure as code solutions, preferably Terraform. Bonus: Cloud certification. Bonus: Production experience with ACID-compliant formats such as Hudi, Iceberg, or Delta Lake. Bonus: Familiarity with data observability solutions and data governance frameworks. Requirements
Bachelor's Degree in Computer Science, Programming, or a related field is preferred. Legal right to work in the USA.
#J-18808-Ljbffr
Collaborate with the team to build out features for the data platform and consolidate data assets. Build, maintain, and optimize data pipelines using Spark. Advise, consult, and coach other data professionals on standards and practices. Work with the team to define company data assets. Migrate CMS' data platform into Chase's environment. Partner with business analysts and solutions architects to develop technical architectures for strategic enterprise projects and initiatives. Build libraries to standardize data processing methods. Demonstrate a passion for teaching and learning, understanding that continuous learning is essential for success. Have a solid understanding of AWS tools such as EMR or Glue, including their advantages and disadvantages, and be able to communicate this knowledge effectively. Implement automation in applicable processes. Mandatory Skills
5+ years of experience in a data engineering role. Proficiency in Python (or similar language) and SQL. Strong experience in building data pipelines with Spark. Excellent verbal and written communication skills. Strong analytical and problem-solving abilities. Experience with relational datastores, NoSQL datastores, and cloud object storage. Experience building data processing infrastructure in AWS. Bonus: Experience with infrastructure as code solutions, preferably Terraform. Bonus: Cloud certification. Bonus: Production experience with ACID-compliant formats such as Hudi, Iceberg, or Delta Lake. Bonus: Familiarity with data observability solutions and data governance frameworks. Requirements
Bachelor's Degree in Computer Science, Programming, or a related field is preferred. Legal right to work in the USA.
#J-18808-Ljbffr