H-E-B
Staff Data Engineer-API Hub/Data Solutions (San Antonio, Austin or Dallas)
H-E-B, San Antonio, Texas, United States, 78208
Staff Data Engineer
Responsibilities Since H-E-B Digital Technology's inception, weve been investing heavily in our customers digital experience, reinventing how they find inspiration from food, make food decisions, and ultimately get food into their homes. This is an exciting time to join H-E-B Digitalwere using the best available technologies to deliver modern, engaging, reliable, and scalable experiences to meet the needs of our growing audience. As a Staff Data Engineer, youll use an advanced analytical, data-driven approach to drive a deep understanding of our fast-changing business and answer real world questions. Youll work with stakeholders to develop a clear understanding of data and data infrastructure needs, resolve complex data-related technical issues, and ensure optimal data design and efficiency. Once youre eligible, youll become an Owner in the company, so were looking for commitment, hard work, and focus on quality and customer service. "Partner-owned" means our most important resourcespeopledrive the innovation, growth, and success that make H-E-B The Greatest Omnichannel Retailing Company. Do you have a: HEART FOR PEOPLE youre willing to facilitate solutions with multiple engineers, provide upward communication, and mentor others? HEAD FOR BUSINESS you consistently demonstrate and uphold the standards of coding, infrastructure, and process? PASSION FOR RESULTS youre capable of high-velocity contributions in multiple technical domains? We are looking for: 8+ years of experience related to data engineering What Youll Do Builds / supports more complex data pipelines, application programming interfaces (APIs), data integrations, data streaming solutions Designs data patterns that support creation of datasets for analytics; implements calculations, cleanses data, ensures standardization of data, maps / links data from more than one source Performs data validation and quality assurance on work of senior peers and write automated tests Maintains / streamlines / orchestrates existing data pipelines end to end Builds large-scale batch and real-time data pipelines with big data processing frameworks Designs / implements monitoring capabilities based on business SLA and data quality Designs / develops / maintains large data pipelines; diagnoses / solves production support issues Uses / contributes to refinement of Digital Engineering-related tools, standards, and training Designs and develops real-time streaming requirements by using structured streaming from either pub-sub or Tibco on datalake Implement features to continuously improve data integration performance Implement Infrastructure as code, security, and CI/CD for data pipelines Engages / collaborates with external technical teams to ensure timely, high-quality solutions Works closely with the application/stakeholder teams to have a clear understanding of data Performs full SDLC process, including planning, design, development, certification, implementation, and support Build strong relationships with cross functional teams to accomplish impactful results. Work with teams such as Data Engineering, Application and Product Managers. Peer reviews with team members; learns / adapts from peer review of own code Contributes to overall design, architecture, security, scalability, reliability, and performance Mentors / provides support to Senior Data Engineers Builds more complex data models to deliver insightful analytics; ensures highest standard in data integrity Has knowledge in machine learning concepts Project You Will Impact Build APIs using FastAPI framework to deliver composite APIs to the applications API service migration from on prem to cloud Ownership of data domains Improve data quality Who You Are 8+ years of hands-on experience related to data engineering in developing data pipelines and APIs Experience with advance SQL, Python and preferred knowledge in Java Proven experience with SQL, Spark, Databricks, AWS Lambda, S3, Data lake Experience of ingesting data from data lake to Elasticsearch/Opensearch Strong knowledge of messaging systems like Kafka, GCP pubsub or Tibco EMS Experience in infrastructure as a code using Terraform Experience in DevOps tools such as GitLab CI/CD, and Jenkins. Experience with orchestration tools such as Argo or Databricks workflow A solid understanding of Big Data and Hybrid Cloud infrastructure. Up to date on latest technological developments. Should be able to evaluate and propose new data pipelines pattern. You have an advanced understanding of SDLC processes. You have a comprehensive knowledge of CS fundamentals: data structures, algorithms, and design patterns. You have advanced knowledge of system architecture and design patterns. You can understand architecture, design, and integration landscape of multiple H-E-B systems. You have experience with common software engineering tools such as Git, JIRA, Confluence, etc. You have a high level of comfort in Lean Startup or Agile development methodologies. You have a related degree or work experience, preferably a Bachelor's degree in related work stream. Excellent written, oral communication and presentation skills. Understanding of Data Engineering Bonus DevOps Certifications Cloud certifications
Responsibilities Since H-E-B Digital Technology's inception, weve been investing heavily in our customers digital experience, reinventing how they find inspiration from food, make food decisions, and ultimately get food into their homes. This is an exciting time to join H-E-B Digitalwere using the best available technologies to deliver modern, engaging, reliable, and scalable experiences to meet the needs of our growing audience. As a Staff Data Engineer, youll use an advanced analytical, data-driven approach to drive a deep understanding of our fast-changing business and answer real world questions. Youll work with stakeholders to develop a clear understanding of data and data infrastructure needs, resolve complex data-related technical issues, and ensure optimal data design and efficiency. Once youre eligible, youll become an Owner in the company, so were looking for commitment, hard work, and focus on quality and customer service. "Partner-owned" means our most important resourcespeopledrive the innovation, growth, and success that make H-E-B The Greatest Omnichannel Retailing Company. Do you have a: HEART FOR PEOPLE youre willing to facilitate solutions with multiple engineers, provide upward communication, and mentor others? HEAD FOR BUSINESS you consistently demonstrate and uphold the standards of coding, infrastructure, and process? PASSION FOR RESULTS youre capable of high-velocity contributions in multiple technical domains? We are looking for: 8+ years of experience related to data engineering What Youll Do Builds / supports more complex data pipelines, application programming interfaces (APIs), data integrations, data streaming solutions Designs data patterns that support creation of datasets for analytics; implements calculations, cleanses data, ensures standardization of data, maps / links data from more than one source Performs data validation and quality assurance on work of senior peers and write automated tests Maintains / streamlines / orchestrates existing data pipelines end to end Builds large-scale batch and real-time data pipelines with big data processing frameworks Designs / implements monitoring capabilities based on business SLA and data quality Designs / develops / maintains large data pipelines; diagnoses / solves production support issues Uses / contributes to refinement of Digital Engineering-related tools, standards, and training Designs and develops real-time streaming requirements by using structured streaming from either pub-sub or Tibco on datalake Implement features to continuously improve data integration performance Implement Infrastructure as code, security, and CI/CD for data pipelines Engages / collaborates with external technical teams to ensure timely, high-quality solutions Works closely with the application/stakeholder teams to have a clear understanding of data Performs full SDLC process, including planning, design, development, certification, implementation, and support Build strong relationships with cross functional teams to accomplish impactful results. Work with teams such as Data Engineering, Application and Product Managers. Peer reviews with team members; learns / adapts from peer review of own code Contributes to overall design, architecture, security, scalability, reliability, and performance Mentors / provides support to Senior Data Engineers Builds more complex data models to deliver insightful analytics; ensures highest standard in data integrity Has knowledge in machine learning concepts Project You Will Impact Build APIs using FastAPI framework to deliver composite APIs to the applications API service migration from on prem to cloud Ownership of data domains Improve data quality Who You Are 8+ years of hands-on experience related to data engineering in developing data pipelines and APIs Experience with advance SQL, Python and preferred knowledge in Java Proven experience with SQL, Spark, Databricks, AWS Lambda, S3, Data lake Experience of ingesting data from data lake to Elasticsearch/Opensearch Strong knowledge of messaging systems like Kafka, GCP pubsub or Tibco EMS Experience in infrastructure as a code using Terraform Experience in DevOps tools such as GitLab CI/CD, and Jenkins. Experience with orchestration tools such as Argo or Databricks workflow A solid understanding of Big Data and Hybrid Cloud infrastructure. Up to date on latest technological developments. Should be able to evaluate and propose new data pipelines pattern. You have an advanced understanding of SDLC processes. You have a comprehensive knowledge of CS fundamentals: data structures, algorithms, and design patterns. You have advanced knowledge of system architecture and design patterns. You can understand architecture, design, and integration landscape of multiple H-E-B systems. You have experience with common software engineering tools such as Git, JIRA, Confluence, etc. You have a high level of comfort in Lean Startup or Agile development methodologies. You have a related degree or work experience, preferably a Bachelor's degree in related work stream. Excellent written, oral communication and presentation skills. Understanding of Data Engineering Bonus DevOps Certifications Cloud certifications