eTeam
Lead Data Engineer - GCP BigQuery, Python & ML Workflows
eTeam, Irving, Texas, United States, 75084
Must Have Skills:
BigQuery, Python, Java
Detailed Job Description: • Minimum 8 years of experience in data engineering or analytics, with 5+ years in GCP BigQuery. • Exposure to machine learning workflows using Vertex AI or BigQuery ML. • Collaborate with data scientists, analysts, and business stakeholders to understand data needs and deliver actionable insights. • Develop and maintain ETL/ELT workflows using Cloud Dataflow, Cloud Composer, or other GCP-native tools. • Experience with GCP services such as Cloud Storage, Pub/Sub, Dataflow, Cloud Functions, and Cloud Composer. • Implement data governance, security, and compliance best practices within GCP. • Familiarity with Python, Java, or Scala for data processing. • Strong proficiency in SQL, BigQuery scripting, and data modeling. • Monitor and troubleshoot performance issues in BigQuery queries and jobs. • Familiarity with CI/CD pipelines and DevOps practices.
BigQuery, Python, Java
Detailed Job Description: • Minimum 8 years of experience in data engineering or analytics, with 5+ years in GCP BigQuery. • Exposure to machine learning workflows using Vertex AI or BigQuery ML. • Collaborate with data scientists, analysts, and business stakeholders to understand data needs and deliver actionable insights. • Develop and maintain ETL/ELT workflows using Cloud Dataflow, Cloud Composer, or other GCP-native tools. • Experience with GCP services such as Cloud Storage, Pub/Sub, Dataflow, Cloud Functions, and Cloud Composer. • Implement data governance, security, and compliance best practices within GCP. • Familiarity with Python, Java, or Scala for data processing. • Strong proficiency in SQL, BigQuery scripting, and data modeling. • Monitor and troubleshoot performance issues in BigQuery queries and jobs. • Familiarity with CI/CD pipelines and DevOps practices.