PETADATA
Position
Senior Data Lake Engineer
Location Dallas, TX (Remote)
Work Type C2C
Experience 15+ Years
Summary Petadata is seeking a seasoned Senior Data Lake Engineer with over fifteen years of experience in data engineering and a strong focus on building and managing AWS‑native Data Lake solutions. The ideal candidate will have deep expertise with AWS Lake Formation, serverless data processing using Lambda and Python, and experience with AI‑assisted development tools such as Amazon Q.
Roles & Responsibilities
Design, build, and optimize scalable, secure data lakes using AWS Lake Formation and best practices for data governance, cataloging, and access control.
Build and deploy AWS Lambda functions using Python for real‑time data processing, automation, and event‑driven workflows.
Develop and maintain robust data pipelines using AWS Glue, integrating data from various structured and unstructured sources.
Leverage AI‑powered coding tools (such as Amazon Q, GitHub Copilot, or similar) to increase development speed, code quality, and automation.
Design and implement integrations between the Data Lake and DynamoDB, optimizing for performance, scale, and consistency.
Implement fine‑grained access control using Lake Formation, IAM policies, encryption, and data masking techniques to meet enterprise and compliance standards (e.g., GDPR, HIPAA).
Implement logging, monitoring, and performance tuning for Glue jobs, Lambda functions, and data workflows.
Collaborate with cross‑functional teams including data science, analytics, DevOps, and product teams. Provide mentorship and technical leadership to junior engineers.
Required Skills
15+ years in data engineering roles, with 5+ years focused on AWS‑native data lake development.
Deep, hands‑on expertise in AWS Lake Formation, Glue, Lambda, and DynamoDB.
Proficient in Python for serverless and data processing applications.
Experience using AI‑assisted development tools (e.g., Amazon Q, GitHub Copilot, AWS CodeWhisperer).
Strong knowledge of data security practices in AWS, including IAM, encryption, and compliance standards.
Experience with workflow orchestration tools such as Step Functions, Airflow, or custom AWS‑based solutions.
Strong communication, problem‑solving, and collaboration skills. Able to lead architecture discussions and best‑practice recommendations.
Preferred Skills
AWS Certifications (e.g., AWS Certified Data Analytics, AWS Certified Solutions Architect)
Experience with Athena, Redshift, or other analytics services in the AWS ecosystem.
Exposure to DevOps practices and tools like Terraform, CloudFormation, or CDK.
Familiarity with data cataloging and metadata management tools.
Education Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
Application Process Please email your resume to
greeshmac@petadata.co . Candidates are required to attend phone/video calls, in‑person interviews, and undergo standard background checks on education and experience. After reviewing your experience and skills, an HR team member will contact you with next steps.
#J-18808-Ljbffr
Location Dallas, TX (Remote)
Work Type C2C
Experience 15+ Years
Summary Petadata is seeking a seasoned Senior Data Lake Engineer with over fifteen years of experience in data engineering and a strong focus on building and managing AWS‑native Data Lake solutions. The ideal candidate will have deep expertise with AWS Lake Formation, serverless data processing using Lambda and Python, and experience with AI‑assisted development tools such as Amazon Q.
Roles & Responsibilities
Design, build, and optimize scalable, secure data lakes using AWS Lake Formation and best practices for data governance, cataloging, and access control.
Build and deploy AWS Lambda functions using Python for real‑time data processing, automation, and event‑driven workflows.
Develop and maintain robust data pipelines using AWS Glue, integrating data from various structured and unstructured sources.
Leverage AI‑powered coding tools (such as Amazon Q, GitHub Copilot, or similar) to increase development speed, code quality, and automation.
Design and implement integrations between the Data Lake and DynamoDB, optimizing for performance, scale, and consistency.
Implement fine‑grained access control using Lake Formation, IAM policies, encryption, and data masking techniques to meet enterprise and compliance standards (e.g., GDPR, HIPAA).
Implement logging, monitoring, and performance tuning for Glue jobs, Lambda functions, and data workflows.
Collaborate with cross‑functional teams including data science, analytics, DevOps, and product teams. Provide mentorship and technical leadership to junior engineers.
Required Skills
15+ years in data engineering roles, with 5+ years focused on AWS‑native data lake development.
Deep, hands‑on expertise in AWS Lake Formation, Glue, Lambda, and DynamoDB.
Proficient in Python for serverless and data processing applications.
Experience using AI‑assisted development tools (e.g., Amazon Q, GitHub Copilot, AWS CodeWhisperer).
Strong knowledge of data security practices in AWS, including IAM, encryption, and compliance standards.
Experience with workflow orchestration tools such as Step Functions, Airflow, or custom AWS‑based solutions.
Strong communication, problem‑solving, and collaboration skills. Able to lead architecture discussions and best‑practice recommendations.
Preferred Skills
AWS Certifications (e.g., AWS Certified Data Analytics, AWS Certified Solutions Architect)
Experience with Athena, Redshift, or other analytics services in the AWS ecosystem.
Exposure to DevOps practices and tools like Terraform, CloudFormation, or CDK.
Familiarity with data cataloging and metadata management tools.
Education Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
Application Process Please email your resume to
greeshmac@petadata.co . Candidates are required to attend phone/video calls, in‑person interviews, and undergo standard background checks on education and experience. After reviewing your experience and skills, an HR team member will contact you with next steps.
#J-18808-Ljbffr