Jobs via Dice
Title
Senior Software Engineer - ETL Developer (Python/Pyspark)
Location Skillman, NJ (Three days onsite is a must)
Job Description Dice is the leading career destination for tech experts at every stage of their careers. Our client, Avenues International, Inc., is seeking the following.
Must have Skills
10+ years of experience in designing and developing ETL pipelines leveraging PySpark/ Python.
Experience on Python database libraries like SQLAlchemy, psycopg2, etc.
Strong understanding of data warehousing methodologies, ETL processing, and dimensional data modeling.
Advanced SQL capabilities required. Knowledge of database design techniques and experience working with extremely large data volumes is a plus.
Demonstrated experience and ability to work with business users to gather requirements and manage scope.
Experience in workflow tools such as Oozie, Airflow, or Tidal.
Experience working in a big data environment with technologies such as Greenplum, Hadoop, and Hive.
BA, BS, MS, PhD in Computer Science, Engineering or related technology field.
Desirable Skills
Experience with large database and DW implementation (20+ TBs).
Understanding of VLDB performance aspects, such as table partitioning, sharding, table distribution, and optimization techniques.
Knowledge of reporting tools such as Qlik Sense, Tableau, Cognos.
Seniority level
Mid-Senior level
Employment type
Full-time
Job function
Information Technology
Industries
Software Development
#J-18808-Ljbffr
Location Skillman, NJ (Three days onsite is a must)
Job Description Dice is the leading career destination for tech experts at every stage of their careers. Our client, Avenues International, Inc., is seeking the following.
Must have Skills
10+ years of experience in designing and developing ETL pipelines leveraging PySpark/ Python.
Experience on Python database libraries like SQLAlchemy, psycopg2, etc.
Strong understanding of data warehousing methodologies, ETL processing, and dimensional data modeling.
Advanced SQL capabilities required. Knowledge of database design techniques and experience working with extremely large data volumes is a plus.
Demonstrated experience and ability to work with business users to gather requirements and manage scope.
Experience in workflow tools such as Oozie, Airflow, or Tidal.
Experience working in a big data environment with technologies such as Greenplum, Hadoop, and Hive.
BA, BS, MS, PhD in Computer Science, Engineering or related technology field.
Desirable Skills
Experience with large database and DW implementation (20+ TBs).
Understanding of VLDB performance aspects, such as table partitioning, sharding, table distribution, and optimization techniques.
Knowledge of reporting tools such as Qlik Sense, Tableau, Cognos.
Seniority level
Mid-Senior level
Employment type
Full-time
Job function
Information Technology
Industries
Software Development
#J-18808-Ljbffr