Compunnel, Inc.
We are seeking a highly energetic and collaborative Senior Data Engineer for a 12-month engagement. The ideal candidate will design and develop big data applications using the latest open-source technologies, automate workflows, create data pipelines, and provide ongoing maintenance and enhancements. This role requires extensive experience in cloud platforms, data engineering, and workflow orchestration.
Key Responsibilities:
Design and develop big data applications using the latest open-source technologies Work in an offshore model and deliver managed outcomes Develop logical and physical data models for big data platforms Automate workflows using Apache Airflow Create data pipelines using Apache Hive, Apache Spark, and Apache Kafka Provide ongoing maintenance and enhancements to existing systems and participate in rotational on-call support Learn business domains and technology infrastructure quickly while sharing knowledge with the team Mentor junior engineers and provide technical guidance Lead daily standups and design reviews Groom and prioritize backlog using JIRA Act as the point of contact for assigned business domains Required Qualifications:
12+ years of hands-on experience in developing data warehouse solutions and data products 6+ years of experience developing distributed data processing platforms with Hadoop, Hive, Spark, and Airflow 5+ years of experience in modeling and designing schemas for data lakes or RDBMS platforms 4+ years of recent hands-on experience with Google Cloud Platform (GCP) Experience building data pipelines in GCP, including GCP Dataproc, GCS, and BigQuery Proficiency in programming languages such as Python, Java, and Scala Experience with scripting languages such as Perl and Shell Expertise in processing and managing large-scale data sets (multi-TB/PB scale) Exposure to test-driven development and automated testing frameworks Background in Scrum/Agile development methodologies Strong ability to manage multiple priorities with minimal supervision Excellent verbal and written communication skills Bachelor's degree in Computer Science or equivalent experience Preferred Qualifications:
Experience with Gitflow and Atlassian products (BitBucket, JIRA, Confluence) Familiarity with Continuous Integration tools such as Bamboo, Jenkins, or TFS Location:
Bentonville, Arkansas, 72712
#J-18808-Ljbffr
Design and develop big data applications using the latest open-source technologies Work in an offshore model and deliver managed outcomes Develop logical and physical data models for big data platforms Automate workflows using Apache Airflow Create data pipelines using Apache Hive, Apache Spark, and Apache Kafka Provide ongoing maintenance and enhancements to existing systems and participate in rotational on-call support Learn business domains and technology infrastructure quickly while sharing knowledge with the team Mentor junior engineers and provide technical guidance Lead daily standups and design reviews Groom and prioritize backlog using JIRA Act as the point of contact for assigned business domains Required Qualifications:
12+ years of hands-on experience in developing data warehouse solutions and data products 6+ years of experience developing distributed data processing platforms with Hadoop, Hive, Spark, and Airflow 5+ years of experience in modeling and designing schemas for data lakes or RDBMS platforms 4+ years of recent hands-on experience with Google Cloud Platform (GCP) Experience building data pipelines in GCP, including GCP Dataproc, GCS, and BigQuery Proficiency in programming languages such as Python, Java, and Scala Experience with scripting languages such as Perl and Shell Expertise in processing and managing large-scale data sets (multi-TB/PB scale) Exposure to test-driven development and automated testing frameworks Background in Scrum/Agile development methodologies Strong ability to manage multiple priorities with minimal supervision Excellent verbal and written communication skills Bachelor's degree in Computer Science or equivalent experience Preferred Qualifications:
Experience with Gitflow and Atlassian products (BitBucket, JIRA, Confluence) Familiarity with Continuous Integration tools such as Bamboo, Jenkins, or TFS Location:
Bentonville, Arkansas, 72712
#J-18808-Ljbffr