vCreaTek Consulting Services Pvt Ltd
Lead Data Engineer
vCreaTek Consulting Services Pvt Ltd - Snowflake, Arizona, United States, 85937
Work at vCreaTek Consulting Services Pvt Ltd
Overview
- View job
Overview
We are seeking a highly skilled and experienced Lead Data Engineer to join our dynamic team.
Responsibilities
Roles and Responsibilities
1. Design and Architecture: Lead the design and implementation of data pipelines using Databricks and Snowflake. Design data lakehouse architectures for efficient data storage and processing. Stay updated on the latest trends and best practices in data engineering. Coding and Development: Write high-quality, efficient, and maintainable code for data pipelines and transformations. Conduct code reviews to ensure code quality, adherence to best practices, and maintainability. 2. Technical Support: Provide technical support and guidance to junior data engineers on Databricks and Snowflake. Troubleshoot data engineering problems and identify root causes. Technical Team Management (No People Management responsibilities) Mentor and guide junior data engineers in their professional development. Foster a collaborative and learning environment within the data engineering team. 3. Hands-on Coding: A lead data engineer in this role is expected to be a strong coder who can contribute to the team's development efforts. This includes: Developing data pipelines and transformations in both Databricks and Snowflake. Writing unit tests for data engineering code to ensure quality and reliability. Staying up-to-date with the latest libraries and frameworks for data engineering By combining technical expertise, strong leadership skills, and a hands-on coding approach, this lead data engineer can effectively guide the team in building and maintaining a robust data infrastructure using Azure Databricks and Snowflake.
Skills Technical Skills:
1. Databricks: In-depth knowledge of Apache Spark and its ecosystem (SQL, Python, Scala) for data processing and analytics. Experience with Databricks notebooks, clusters, and runtime configuration. Familiarity with Delta Lake and its functionalities. 2. Snowflake: Expertise in writing SQL queries for data warehousing and analytics. Understanding of Snowflake's architecture, including virtual warehouses and schemas. Experience with data ingestion and transformation techniques in Snowflake. 3. Cloud Technologies: Familiarity with Azure cloud services for Databricks deployments (optional). 4. Programming: Strong proficiency in Python, Scala, or SQL for data manipulation and analysis. Experience with data engineering libraries (e.g., Pandas, Koalas). Version Control Systems: Proficiency with Git for code version control and collaboration. Soft Skills: 1. Communication: Excellent written and verbal communication skills to collaborate with technical and non-technical audiences. 2. Leadership: Ability to mentor and guide junior data engineers. 3. Problem-solving: Strong analytical and problem-solving skills to tackle complex data engineering challenges. 4. Teamwork Ability to work effectively in a team environment and foster collaboration.
Agree * I agree to share my personal information as vCreaTek is committed to safeguarding the privacy of your personal information and will only use it for legitimate business purposes related to recruitment and hiring processes. Your information will be treated confidentially and will not be shared with third parties without your consent, except as required by law. * Required fields Phone Thank you! Your profile has been shared with HR. We provide impactful Analytical & Visualization Services.
We have diverse offerings integrated with the latest technical insights that would burgeon exciting opportunities for you and your business! USA
Delivery Center VCREATEK CONSULTING INC 1 Lincoln Highway,Suite 17, Edison,NJ. 08820 UK
Sales Office VCREATEK CONSULTING 234, Century Warf, Chantlery, Cardiff CF10 5NQ Copyright 2024 by vCreaTek LLC All Right Reserved. Registered Office: VCREATEK CONSULTING SERVICES PVT LTD C-9 Hermes Drome. Viman Nagar, Pune, Maharashtra 411014, India #J-18808-Ljbffr
1. Design and Architecture: Lead the design and implementation of data pipelines using Databricks and Snowflake. Design data lakehouse architectures for efficient data storage and processing. Stay updated on the latest trends and best practices in data engineering. Coding and Development: Write high-quality, efficient, and maintainable code for data pipelines and transformations. Conduct code reviews to ensure code quality, adherence to best practices, and maintainability. 2. Technical Support: Provide technical support and guidance to junior data engineers on Databricks and Snowflake. Troubleshoot data engineering problems and identify root causes. Technical Team Management (No People Management responsibilities) Mentor and guide junior data engineers in their professional development. Foster a collaborative and learning environment within the data engineering team. 3. Hands-on Coding: A lead data engineer in this role is expected to be a strong coder who can contribute to the team's development efforts. This includes: Developing data pipelines and transformations in both Databricks and Snowflake. Writing unit tests for data engineering code to ensure quality and reliability. Staying up-to-date with the latest libraries and frameworks for data engineering By combining technical expertise, strong leadership skills, and a hands-on coding approach, this lead data engineer can effectively guide the team in building and maintaining a robust data infrastructure using Azure Databricks and Snowflake.
Skills Technical Skills:
1. Databricks: In-depth knowledge of Apache Spark and its ecosystem (SQL, Python, Scala) for data processing and analytics. Experience with Databricks notebooks, clusters, and runtime configuration. Familiarity with Delta Lake and its functionalities. 2. Snowflake: Expertise in writing SQL queries for data warehousing and analytics. Understanding of Snowflake's architecture, including virtual warehouses and schemas. Experience with data ingestion and transformation techniques in Snowflake. 3. Cloud Technologies: Familiarity with Azure cloud services for Databricks deployments (optional). 4. Programming: Strong proficiency in Python, Scala, or SQL for data manipulation and analysis. Experience with data engineering libraries (e.g., Pandas, Koalas). Version Control Systems: Proficiency with Git for code version control and collaboration. Soft Skills: 1. Communication: Excellent written and verbal communication skills to collaborate with technical and non-technical audiences. 2. Leadership: Ability to mentor and guide junior data engineers. 3. Problem-solving: Strong analytical and problem-solving skills to tackle complex data engineering challenges. 4. Teamwork Ability to work effectively in a team environment and foster collaboration.
Agree * I agree to share my personal information as vCreaTek is committed to safeguarding the privacy of your personal information and will only use it for legitimate business purposes related to recruitment and hiring processes. Your information will be treated confidentially and will not be shared with third parties without your consent, except as required by law. * Required fields Phone Thank you! Your profile has been shared with HR. We provide impactful Analytical & Visualization Services.
We have diverse offerings integrated with the latest technical insights that would burgeon exciting opportunities for you and your business! USA
Delivery Center VCREATEK CONSULTING INC 1 Lincoln Highway,Suite 17, Edison,NJ. 08820 UK
Sales Office VCREATEK CONSULTING 234, Century Warf, Chantlery, Cardiff CF10 5NQ Copyright 2024 by vCreaTek LLC All Right Reserved. Registered Office: VCREATEK CONSULTING SERVICES PVT LTD C-9 Hermes Drome. Viman Nagar, Pune, Maharashtra 411014, India #J-18808-Ljbffr