Argyll Infotech Enterprise Pvt Ltd
Lead Data Engineer
Argyll Infotech Enterprise Pvt Ltd, San Jose, California, United States, 95199
Hello Everyone,
No GC,
Role:
Senior Lead Data Engineer
Location:
San Jose, CA 95125 Hybrid (Locals)
Key:
Hadoop, Spark SQL strong
Base pay range $60.00/yr - $65.00/yr
Key Responsibilities
Architect, construct, test and maintain ELT pipelines using Hadoop and Spark SQL.
Manage and optimize the infrastructure to ensure reliability and performance.
Design and implement data models and schemas for enterprise data warehouse and data marts.
Write and maintain scripts in Python for various applications.
Develop, optimize, troubleshoot complex SQL queries and database structures for relational and non-relational databases.
Work closely with analysts and product managers to understand data requirements and deliver high‑quality data.
Provide technical guidance and mentorship to other team members, fostering best practices in data engineering.
Monitor system performance, identify bottlenecks, and implement solutions to ensure high data availability.
Required Skills
High proficiency in the Hadoop ecosystem (HDFS, MapReduce, YARN) and Apache Spark, Scala for large‑scale data processing.
Strong Python skills for data manipulation and scripting.
Mastery of advanced SQL and deep experience with relational databases and data warehouses.
Hands‑on experience with workflow management tools like Apache Airflow.
Familiarity with Unix shell scripting for automation and system management.
Seniority level Mid‑Senior level
Employment type Contract
Job function Information Technology
Industries IT Services and IT Consulting
#J-18808-Ljbffr
No GC,
Role:
Senior Lead Data Engineer
Location:
San Jose, CA 95125 Hybrid (Locals)
Key:
Hadoop, Spark SQL strong
Base pay range $60.00/yr - $65.00/yr
Key Responsibilities
Architect, construct, test and maintain ELT pipelines using Hadoop and Spark SQL.
Manage and optimize the infrastructure to ensure reliability and performance.
Design and implement data models and schemas for enterprise data warehouse and data marts.
Write and maintain scripts in Python for various applications.
Develop, optimize, troubleshoot complex SQL queries and database structures for relational and non-relational databases.
Work closely with analysts and product managers to understand data requirements and deliver high‑quality data.
Provide technical guidance and mentorship to other team members, fostering best practices in data engineering.
Monitor system performance, identify bottlenecks, and implement solutions to ensure high data availability.
Required Skills
High proficiency in the Hadoop ecosystem (HDFS, MapReduce, YARN) and Apache Spark, Scala for large‑scale data processing.
Strong Python skills for data manipulation and scripting.
Mastery of advanced SQL and deep experience with relational databases and data warehouses.
Hands‑on experience with workflow management tools like Apache Airflow.
Familiarity with Unix shell scripting for automation and system management.
Seniority level Mid‑Senior level
Employment type Contract
Job function Information Technology
Industries IT Services and IT Consulting
#J-18808-Ljbffr