Data Engineer

Key Responsibilities Data Pipeline Development Design, develop, and maintain robust ETL/ELT pipelines using Databricks and AWS services (Glue, Lambda, S3, EMR). Write efficient, maintainable code in Python for data ingestion, transformation, and automation. Database & SQL Mastery Use advanced SQL skills to perform complex queries, data transformations, and performance tuning on RDS/Postgres. Create and optimize database structures (tables, indexes, partitions) to support analytics and reporting needs. AWS Architecture & Infrastructure Leverage AWS best practices to architect secure, scalable, and high-performing cloud data environments (EC2, S3, Lambda, IAM, VPC, etc.). Provide Solution Architect-level guidance, ensuring alignment with best practices for cost optimization, security, and reliability. Performance Optimization & Troubleshooting Continuously monitor the health and performance of data pipelines and databases. Diagnose and resolve bottlenecks in data ingestion, transformation, and querying processes. Collaboration & Stakeholder Management Work closely with data scientists, data analysts, and business stakeholders to gather requirements and deliver data-driven solutions. Communicate progress, roadblocks, and technical details to both technical and non-technical team members. Data Governance & Security Implement and maintain data governance protocols, ensuring compliance with industry regulations (GDPR, HIPAA, etc., as relevant). Establish security best practices for data access, encryption, backup, and disaster recovery. Qualifications & Skills Required: Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field (or equivalent work experience). Expert-level proficiency in Python for data engineering tasks (ETL, scripting, automation). Advanced SQL skills, including performance tuning, complex queries, stored procedures, and data transformations. Hands-on experience with Databricks for data pipeline development and orchestration. Strong understanding of AWS services (EC2, S3, Lambda, Glue, IAM, VPC) and familiarity with Solution Architect practices. Proven track record of RDS/Postgres administration, including performance optimization and schema design. Familiarity with version control (Git) and CI/CD pipelines. Preferred: AWS certifications (e.g., AWS Certified Solutions Architect – Associate or Professional). Experience with other data warehouse technologies (Snowflake) and BI/analytics tools. Exposure to infrastructure-as-code (CloudFormation, Terraform) and containerization (Docker, Kubernetes). Knowledge of Java at a moderate level for specific data-related use cases or integrations. Soft Skills Exceptional problem-solving capabilities and a strong attention to detail. Effective communication and collaboration skills across technical and non-technical teams. Ability to work independently, set priorities, and manage multiple projects in a fast-paced environment. Eagerness to learn and adapt in an ever-evolving tech landscape. Role Description We are seeking a Data Engineer with expert-level Python and SQL skills. The ideal candidate will have experience designing and building cloud-based data solutions in AWS (preferably with Solution Architect expertise), Databricks, and RDS/Postgres. In this role, you will be responsible for creating and optimizing scalable data pipelines, ensuring the integrity and performance of our data infrastructure, and collaborating with cross-functional teams to drive business insights.

#J-18808-Ljbffr