Compunnel, Inc.
We are seeking an experienced AWS Data Engineer to design, build, and maintain scalable data pipelines and architectures on the AWS cloud platform.
This role involves developing ETL/ELT processes, integrating data from multiple sources, ensuring data quality and security, and collaborating with stakeholders to deliver optimized data solutions for analytics and business intelligence.
Key Responsibilities
Design and build data pipelines and ETL/ELT processes using AWS services such as Glue, EMR, and Redshift. Develop and maintain efficient data models for storage and analysis. Integrate data from various sources and ensure data quality, security, and compliance. Implement validation checks and security best practices across data workflows. Monitor and optimize data processing jobs and databases for performance. Collaborate with stakeholders to understand data requirements and deliver solutions. Maintain and operationalize existing data solutions and support modernization initiatives. Required Qualifications
Minimum 6+ years of experience in designing and developing enterprise-wide big data solutions. Strong experience with AWS services: Glue, Lambda, S3, EMR, SNS/SQS, CloudWatch, Redshift. Proficiency in Scala and Python for application development and automation. Strong SQL experience, preferably with Redshift; experience with relational databases (Oracle, MySQL, PostgreSQL). Hands‑on experience with ETL/ELT processes and frameworks. Experience with Big Data technologies such as Hadoop and Spark. Familiarity with shell scripting in Linux/Unix environments. Preferred Qualifications
Experience with Snowflake. Knowledge of CI/CD pipelines and automation tools. Exposure to machine learning models, regression, and validation techniques.
#J-18808-Ljbffr
Design and build data pipelines and ETL/ELT processes using AWS services such as Glue, EMR, and Redshift. Develop and maintain efficient data models for storage and analysis. Integrate data from various sources and ensure data quality, security, and compliance. Implement validation checks and security best practices across data workflows. Monitor and optimize data processing jobs and databases for performance. Collaborate with stakeholders to understand data requirements and deliver solutions. Maintain and operationalize existing data solutions and support modernization initiatives. Required Qualifications
Minimum 6+ years of experience in designing and developing enterprise-wide big data solutions. Strong experience with AWS services: Glue, Lambda, S3, EMR, SNS/SQS, CloudWatch, Redshift. Proficiency in Scala and Python for application development and automation. Strong SQL experience, preferably with Redshift; experience with relational databases (Oracle, MySQL, PostgreSQL). Hands‑on experience with ETL/ELT processes and frameworks. Experience with Big Data technologies such as Hadoop and Spark. Familiarity with shell scripting in Linux/Unix environments. Preferred Qualifications
Experience with Snowflake. Knowledge of CI/CD pipelines and automation tools. Exposure to machine learning models, regression, and validation techniques.
#J-18808-Ljbffr