Compunnel
Job Summary
We are seeking a highly experienced Data Engineer Lead to design, develop, and maintain robust data engineering solutions.
The ideal candidate will have deep expertise in ETL/ELT processes, big data technologies, and cloud-based data platforms.
This role requires strong technical leadership, hands-on development skills, and a solid understanding of the financial services domain.
Key Responsibilities Design and implement scalable ETL/ELT processes and data pipelines. Develop and maintain data engineering solutions using Python and Scala. Manage and optimize relational databases such as Oracle, MySQL, and PostgreSQL. Utilize big data technologies including Hadoop and Spark for large-scale data processing. Implement CI/CD pipelines and manage code repositories using Git. Create technical design documentation and conduct peer reviews of functional designs. Perform unit testing, code reviews, and ensure adherence to development standards. Troubleshoot and resolve complex issues during development and testing phases. Drive performance testing and support production environments. Design and develop big data solutions using AWS services such as Glue, Lambda, SNS/SQS, and CloudWatch. Work with Redshift and Snowflake for data warehousing and analytics. Required Qualifications
8+ years of experience with ETL/ELT processes and frameworks. 8+ years of expertise in SQL and relational databases (Oracle, MySQL, PostgreSQL). 5+ years of hands-on experience with Python and Scala for data engineering. 8+ years of experience with Git and CI/CD deployment processes. 5+ years of experience with big data technologies (Hadoop, Spark). Strong experience in application development using Python. Experience with AWS data services (Glue, Lambda, SNS/SQS, CloudWatch). Proficiency in Redshift and Snowflake. Strong documentation, testing, and troubleshooting skills. Preferred Qualifications
5–8 years of experience in one or more of the following domains: Asset Management Financial Services Investment Management Hedge Funds
Education:
Bachelors Degree
We are seeking a highly experienced Data Engineer Lead to design, develop, and maintain robust data engineering solutions.
The ideal candidate will have deep expertise in ETL/ELT processes, big data technologies, and cloud-based data platforms.
This role requires strong technical leadership, hands-on development skills, and a solid understanding of the financial services domain.
Key Responsibilities Design and implement scalable ETL/ELT processes and data pipelines. Develop and maintain data engineering solutions using Python and Scala. Manage and optimize relational databases such as Oracle, MySQL, and PostgreSQL. Utilize big data technologies including Hadoop and Spark for large-scale data processing. Implement CI/CD pipelines and manage code repositories using Git. Create technical design documentation and conduct peer reviews of functional designs. Perform unit testing, code reviews, and ensure adherence to development standards. Troubleshoot and resolve complex issues during development and testing phases. Drive performance testing and support production environments. Design and develop big data solutions using AWS services such as Glue, Lambda, SNS/SQS, and CloudWatch. Work with Redshift and Snowflake for data warehousing and analytics. Required Qualifications
8+ years of experience with ETL/ELT processes and frameworks. 8+ years of expertise in SQL and relational databases (Oracle, MySQL, PostgreSQL). 5+ years of hands-on experience with Python and Scala for data engineering. 8+ years of experience with Git and CI/CD deployment processes. 5+ years of experience with big data technologies (Hadoop, Spark). Strong experience in application development using Python. Experience with AWS data services (Glue, Lambda, SNS/SQS, CloudWatch). Proficiency in Redshift and Snowflake. Strong documentation, testing, and troubleshooting skills. Preferred Qualifications
5–8 years of experience in one or more of the following domains: Asset Management Financial Services Investment Management Hedge Funds
Education:
Bachelors Degree