Python + PySpark - Junior Developer

The Python + PySpark + AWS Consultant will be responsible for designing, developing, and optimizing data engineering solutions using Python and PySpark. The role requires hands-on experience in building scalable ETL pipelines, working with cloud platforms such as AWS or Azure, and ensuring data integrity and performance optimization. The candidate should have strong SQL skills and a deep understanding of data engineering principles. Key Responsibilities • Develop and maintain data processing workflows using Python and PySpark • Design and implement ETL pipelines for structured and unstructured data • Optimize Spark-based data processing for efficiency and scalability • Deploy and manage data solutions on AWS or Azure • Write and optimize SQL queries for data transformation and analysis • Troubleshoot and resolve performance issues in data pipelines • Work closely with cross-functional teams to ensure data reliability and integrity Required Qualifications • 5+ years of experience in data engineering • Strong proficiency in Python and object-oriented programming • Hands-on experience with PySpark for large-scale data processing • Proficiency in SQL for data manipulation and query performance tuning • Experience with AWS or Azure for cloud-based data solutions • Knowledge of ETL processes and data pipeline automation • Experience with Hadoop is acceptable Preferred Qualifications • Experience in optimizing Spark jobs for performance and cost efficiency • Familiarity with DevOps practices for data engineering • Understanding of data governance, security, and compliance best practices

#J-18808-Ljbffr