Accord Technologies Inc
Senior Lead AWS Data Engineer
Accord Technologies Inc, Mooresville, North Carolina, United States, 28115
Overview
Lead the design, implementation, and optimization of ETL workflows using Python and Spark. Build and maintain state-of-the-art data platforms on AWS, contributing to a robust Lakehouse architecture. Serve as a hands-on technical leader, guiding best practices for data engineering solutions and development standards. Responsibilities
Lead the design, implementation, and optimization of ETL workflows using Python and Spark. Build and maintain state-of-the-art data platforms on AWS, contributing to a robust Lakehouse architecture. Serve as a hands-on technical leader, guiding best practices for data engineering solutions and development standards. Work closely with architects, Product Owners, and development teams to decompose projects into Epics and plan technical components. Drive the migration of existing data workflows to a Lakehouse architecture, utilizing technologies like Iceberg. Implement data pipelines using AWS services such as EMR, Glue, Lambda, Step Functions, API Gateway, and Athena. Collaborate within an Agile team environment, ensuring alignment and clear communication of complex technical concepts. Ensure comprehensive and accessible technical documentation for knowledge sharing and compliance. Uphold quality assurance through code reviews, automated testing, and data validation best practices. Leverage automation and CI/CD pipelines to streamline development, testing, and deployment processes. Remain adaptable to project-specific technology needs, with deep expertise in Python and Spark. (Bonus) Apply financial services experience, particularly with equity, fixed income, and index data, if available. (Bonus) Utilize solution architecture experience and relevant certifications (e.g., AWS Solutions Architect, Data Analytics). Seniority level
Mid-Senior level Employment type
Contract Job function
Information Technology
#J-18808-Ljbffr
Lead the design, implementation, and optimization of ETL workflows using Python and Spark. Build and maintain state-of-the-art data platforms on AWS, contributing to a robust Lakehouse architecture. Serve as a hands-on technical leader, guiding best practices for data engineering solutions and development standards. Responsibilities
Lead the design, implementation, and optimization of ETL workflows using Python and Spark. Build and maintain state-of-the-art data platforms on AWS, contributing to a robust Lakehouse architecture. Serve as a hands-on technical leader, guiding best practices for data engineering solutions and development standards. Work closely with architects, Product Owners, and development teams to decompose projects into Epics and plan technical components. Drive the migration of existing data workflows to a Lakehouse architecture, utilizing technologies like Iceberg. Implement data pipelines using AWS services such as EMR, Glue, Lambda, Step Functions, API Gateway, and Athena. Collaborate within an Agile team environment, ensuring alignment and clear communication of complex technical concepts. Ensure comprehensive and accessible technical documentation for knowledge sharing and compliance. Uphold quality assurance through code reviews, automated testing, and data validation best practices. Leverage automation and CI/CD pipelines to streamline development, testing, and deployment processes. Remain adaptable to project-specific technology needs, with deep expertise in Python and Spark. (Bonus) Apply financial services experience, particularly with equity, fixed income, and index data, if available. (Bonus) Utilize solution architecture experience and relevant certifications (e.g., AWS Solutions Architect, Data Analytics). Seniority level
Mid-Senior level Employment type
Contract Job function
Information Technology
#J-18808-Ljbffr