Compunnel, Inc.
GCP Data Engineer
Compunnel, Inc. - Richardson, Texas, United States, 75080
Work at Compunnel, Inc.
Overview
- View job
Overview
We are seeking an experienced GCP Data Engineer with strong experience in building and optimizing big data pipelines on Google Cloud Platform (GCP). The ideal candidate will have hands-on experience with GCP services, big data frameworks, and Python-based ETL development, and will contribute to building robust data solutions that scale to meet complex analytics requirements.
Job Responsibilities:
Design, develop, and maintain scalable data pipelines and workflows using GCP services such as Dataflow, Cloud Composer, and Cloud Functions
Implement ETL/ELT solutions using PySpark, Spark SQL, and Python to process and transform large datasets
Develop monitoring and alerting mechanisms for data quality and pipeline failures
Write advanced SQL queries to support reporting and analytics use cases
Work with GCP services like Compute Engine, DataProc, Kubernetes Engine, BigQuery, Pub/Sub, and Cloud Storage
Participate in CI/CD pipeline automation using tools like Jenkins, GitHub, and GitHub Actions
Design and implement data models (conceptual, logical, and physical) for analytics and reporting
Conduct architecture and code reviews to ensure solutions align with GCP best practices and performance standards
Provide architectural recommendations for scalable and secure cloud data solution
Collaborate with cross-functional teams across data engineering, data science, and product teams
Required Skills:
Strong hands-on experience with core GCP services: Compute Engine, DataProc, Kubernetes Engine, Cloud Storage, BigQuery, Pub/Sub, Cloud Functions, and Dataflow Proficiency in PySpark, Python, Spark SQL, DataFrames, and PyTest Experience with Cloud Composer (Airflow on GCP) for orchestrating data workflows Proven expertise in writing complex and optimized SQL queries Experience designing and implementing CI/CD pipelines using GitHub, GitHub Actions, and Jenkins Deep understanding of data architecture, data modeling, and pipeline optimization on GCP Strong troubleshooting and debugging skills Excellent verbal and written communication skills Preferred Skills:
Experience with data governance and metadata management tools Familiarity with machine learning pipelines on GCP GCP Professional Data Engineer certification is a plus Certifications:
GCP Professional Data Engineer (preferred) Education: Bachelor's or master’s degree in computer science, Engineering, Data Science, or a related field Email ID * This field is required Please enter valid emailId. Cell phone * This field is required Please enter valid cell phone. First Name * This field is required Please enter valid first name. Last Name * This field is required Please enter valid last name.
#J-18808-Ljbffr
Strong hands-on experience with core GCP services: Compute Engine, DataProc, Kubernetes Engine, Cloud Storage, BigQuery, Pub/Sub, Cloud Functions, and Dataflow Proficiency in PySpark, Python, Spark SQL, DataFrames, and PyTest Experience with Cloud Composer (Airflow on GCP) for orchestrating data workflows Proven expertise in writing complex and optimized SQL queries Experience designing and implementing CI/CD pipelines using GitHub, GitHub Actions, and Jenkins Deep understanding of data architecture, data modeling, and pipeline optimization on GCP Strong troubleshooting and debugging skills Excellent verbal and written communication skills Preferred Skills:
Experience with data governance and metadata management tools Familiarity with machine learning pipelines on GCP GCP Professional Data Engineer certification is a plus Certifications:
GCP Professional Data Engineer (preferred) Education: Bachelor's or master’s degree in computer science, Engineering, Data Science, or a related field Email ID * This field is required Please enter valid emailId. Cell phone * This field is required Please enter valid cell phone. First Name * This field is required Please enter valid first name. Last Name * This field is required Please enter valid last name.
#J-18808-Ljbffr