AWS Data Engineer

Role : AWS Data Engineer Location : Torrance, CA Duration : Longterm Contract Job Description: 5 years of experience in data engineering, database design, ETL processes, and data warehousing. 3 years of experience with AWS tools and technologies (S3, EMR, Glue, Athena, RedShift, RDS, Spectrum and Airflow) 2 years of experience with CI/CD tools. Strong knowledge of data storage and processing technologies, including databases and data lakes based distributed computing frameworks (e.g., Hadoop, Spark). 3 in programming languages such as Python, Java, or Scala. Nice to have Informatica Cloud experience (IDMC) Detailed Responsibilities: Develop and Maintain Data Integration Solutions: o Design and implement data integration workflows using AWS Glue/EMR,AWS MWAA(Airflow), Lambda, Redshift o Demonstrate proficiency in Pyspark, Apache Spark and Python for data processing large datasets o Ensure data is accurately and efficiently extracted, transformed, and loaded into target systems. Ensure Data Quality and Integrity: o Validate and cleanse data to maintain high data quality. o Ensure data quality and integrity by implementing monitoring, validation, and error handling mechanisms within data pipelines Optimize Data Integration Processes: o Enhance the performance, optimization of data workflows to meet SLAs, scalability of data integration processes and cost-efficiency on AWS cloud infrastructure. o Identify and resolve performance bottlenecks, fine-tuning queries, and optimizing data processing to enhance Redshift's performance o Regularly review and refine integration processes to improve efficiency. Support Business Intelligence and Analytics: o Translate business requirements to technical specifications and coded data pipelines o Ensure timely availability of integrated data for business intelligence and analytics. o Collaborate with data analysts and business stakeholders to meet their data requirements. Maintain Documentation and Compliance: o Document all data integration processes, workflows, and technical & system specifications. o Ensure compliance with data governance policies, industry standards, and regulatory requirements.aa415a4b-8b21-40fc-a65c-70d2b25ca29a