Get new jobs for this search by email

We are looking for an Expert-Level Python Developer with a strong background in AWS Glue, PySpark, and data engineering workflows. You will be responsible for building scalable ETL pipelines, performing data transformation and deconstruction, and delivering cloud-native data solutions. This role requires a deep understanding of distributed computing, cloud automation, and complex data processing in large-scale environments. Key Responsibilities: Design, develop, and optimize ETL workflows using AWS Glue (Spark and Python Shell Jobs) and PySpark. Build and manage scalable data pipelines for structured, semi-structured, and unstructured data. Develop robust Python scripts for data ingestion, validation, transformation, and orchestration. Perform data deconstruction and normalization to convert raw datasets into meaningful, usable formats. Leverage AWS services (S3, Lambda, Step Functions, Athena, Redshift, DynamoDB, CloudWatch) to deliver reliable and secure data solutions. Implement data quality checks, schema evolution handling, and metadata management. Collaborate with data architects, analysts, and business teams to understand data requirements and translate them into technical deliverables. Optimize performance of distributed processing jobs and troubleshoot failures in production pipelines. Maintain documentation, version control, and test coverage for ETL code and workflows. Required Skills & Qualifications: 8+ years of professional experience in Python development. 3+ years of experience with AWS Glue (PySpark and Python Shell jobs). Strong hands-on experience with PySpark and Spark-based transformations. In-depth knowledge of data deconstruction, parsing nested data formats (JSON, XML, Parquet, Avro), and schema mapping. Advanced knowledge of AWS services related to data engineering (S3, Glue, Lambda, Step Functions, CloudWatch, Athena, Redshift). Proficient in writing performant, reusable Python scripts for ETL orchestration and automation. Experience working with large-scale distributed data systems and optimizing Spark job performance. Familiarity with version control systems (e.g., Git) and CI/CD pipelines. Strong SQL skills and experience working with both relational and NoSQL databases. Excellent analytical, debugging, and communication skills. Preferred Qualifications: AWS Certification: Data Analytics Specialty or Solutions Architect Associate/Professional. Experience with Airflow, Glue Workflows, or other job orchestration frameworks. Familiarity with data lakehouse architectures (e.g., using Delta Lake or Iceberg). Understanding of data governance, lineage, and cataloging (e.g., AWS Glue Data Catalog). Exposure to DevOps practices and Infrastructure-as-Code (Terraform, CDK, or CloudFormation). To apply, simply click the "Apply" button or send your updated profile to recruit@ EA Licence No.:18S9405 / EA Reg. No.:R1330864 Percept Solutions is expanding and actively seeking talented individuals. We encourage applicants to follow Percept Solutions on LinkedIn at to stay informed about new opportunities and events. #J-18808-Ljbffr