Compunnel Inc.
Get AI-powered advice on this job and more exclusive features.
Direct message the job poster from Compunnel Inc.
Responsibilities
Work closely with various business, IT, Analyst and Data Science groups to collect business requirements.
Design, develop, deploy and support high performance data pipelines both inbound and outbound.
Model data platform by applying the business logic and building objects in the semantic layer of the data platform.
Optimize data pipelines for performance, scalability, and reliability.
Implement CI/CD pipelines to ensure continuous deployment and delivery of our data products.
Ensure quality of critical data elements, prepare data quality remediation plans and collaborate with business and system owners to fix the quality issues at its root.
Document the design and support strategy of the data pipelines.
Capture, store and socialize data lineage and operational metadata.
Troubleshoot and resolve data engineering issues as they arise.
Develop REST APIs to expose data to other teams within the company.
Mentor and guide junior data engineers.
General requirements
Performs other duties as assigned.
Complies with all policies and standards.
Qualifications Education
Required - Bachelor’s degree in Computer Science, Computer Engineering, Software Engineering, or other related technical field
Nice to Have - Master’s Degree in Computer Science, Computer Engineering, Software Engineering, or other related technical field
Work Experience
Minimum 6 years of experience in data engineering solutions such as data platforms, ingestion, data management, or publication/analytics
2 years of experience in Google Cloud with services like BigQuery, Composer, GCS, DataStream, Dataflows
Knowledge, Skills and Abilities
Expert knowledge on SQL and Python programming
Experience working with Airflow as workflow management tools and build operators to connect, extract and ingest data as needed
Experience in tuning queries for performance and scalability
Experience in Real Time Data ingestion using GCP PubSub, Kafka, Spark or similar
Excellent organizational, prioritization and analytical abilities
Have proven experience working in incremental execution through successful launches
Excellent problem-solving and critical-thinking skills to recognize and comprehend complex data issues affecting the business environment
Experience working in agile environment
Seniority level
Mid-Senior level
Employment type
Full-time
Job function
Information Technology
Industries
IT Services and IT Consulting
Referrals increase your chances of interviewing at Compunnel Inc. by 2x
Get notified about new Data Engineer jobs in
Columbia, MD .
We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
#J-18808-Ljbffr
Responsibilities
Work closely with various business, IT, Analyst and Data Science groups to collect business requirements.
Design, develop, deploy and support high performance data pipelines both inbound and outbound.
Model data platform by applying the business logic and building objects in the semantic layer of the data platform.
Optimize data pipelines for performance, scalability, and reliability.
Implement CI/CD pipelines to ensure continuous deployment and delivery of our data products.
Ensure quality of critical data elements, prepare data quality remediation plans and collaborate with business and system owners to fix the quality issues at its root.
Document the design and support strategy of the data pipelines.
Capture, store and socialize data lineage and operational metadata.
Troubleshoot and resolve data engineering issues as they arise.
Develop REST APIs to expose data to other teams within the company.
Mentor and guide junior data engineers.
General requirements
Performs other duties as assigned.
Complies with all policies and standards.
Qualifications Education
Required - Bachelor’s degree in Computer Science, Computer Engineering, Software Engineering, or other related technical field
Nice to Have - Master’s Degree in Computer Science, Computer Engineering, Software Engineering, or other related technical field
Work Experience
Minimum 6 years of experience in data engineering solutions such as data platforms, ingestion, data management, or publication/analytics
2 years of experience in Google Cloud with services like BigQuery, Composer, GCS, DataStream, Dataflows
Knowledge, Skills and Abilities
Expert knowledge on SQL and Python programming
Experience working with Airflow as workflow management tools and build operators to connect, extract and ingest data as needed
Experience in tuning queries for performance and scalability
Experience in Real Time Data ingestion using GCP PubSub, Kafka, Spark or similar
Excellent organizational, prioritization and analytical abilities
Have proven experience working in incremental execution through successful launches
Excellent problem-solving and critical-thinking skills to recognize and comprehend complex data issues affecting the business environment
Experience working in agile environment
Seniority level
Mid-Senior level
Employment type
Full-time
Job function
Information Technology
Industries
IT Services and IT Consulting
Referrals increase your chances of interviewing at Compunnel Inc. by 2x
Get notified about new Data Engineer jobs in
Columbia, MD .
We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
#J-18808-Ljbffr