Comcast
Big Data Engineer
Provide technical leadership within a team responsible for creating and maintaining large scale Big Data systems and Data Environments, used to ingest and process large data sets and provide actionable recommendations using data warehousing and business intelligence; use Linux OS; design, develop, test, and maintain AdTech software that extracts, transforms, and loads large volumes of data; develop software systems in an Agile development environment and on big data platforms, including Apache Spark, AWS, Flink, and Hadoop, using SQL, Presto, Scala, GoLang, Java, and Python; use AWS services including EC2, Lambda, S3, and Route 53; use Datadog and Grafana for monitoring; use Jenkins for CI/CD; develop Microservices API systems to support overall product development; store relational data in MySQL to support API and data processing applications; debug functional and performance issues on big data platforms and software modules; concurrently execute data processing software using multithreading; write code and scripts to extract MRM ad logs from FreeWheel Big Data platforms, and load MRM ad logs and campaign data; perform audience targeting; ingest audience, identity, and segment data; create dashboards and monitors on Datadog to ensure 24x7 availability of critical software deployments; build new software products and web frontend frameworks; analyze product specifications, write technical specs, create monitoring dashboards, develop test suites, design workflows, and setup database schemas and tables; interface with global engineering, operations, services, and business operations teams to execute proof of concepts and incorporate new requirements; improve system performance and ensure availability and scalability of services; provide production support for data processing systems running on AWS cloud and Snowflake; troubleshoot data processing problems running on distributed systems; and guide and mentor junior-level engineers. Position is eligible to work remotely within normal commuting distance of the worksite. Bachelor's degree, or foreign equivalent, in Computer Science, Engineering, or related technical field, and five (5) years of experience developing software in an Agile development environment using SQL, Presto, Scala, GoLang, Java, and Python on Big Data platforms including Apache Spark, AWS, Flink, and Hadoop; using Linux OS; using AWS services including EC2, Lambda, S3, and Route 53; using Datadog or Grafana for monitoring; using Jenkins for CI/CD; storing relational data in MySQL to support data processing applications; and debugging functional and performance issues on big data platforms and software modules. Disclaimer: This information has been designed to indicate the general nature and level of work performed by employees in this role. It is not designed to contain or be interpreted as a comprehensive inventory of all duties, responsibilities and qualifications. Skills Agile Environments, Apache Spark, Presto
Provide technical leadership within a team responsible for creating and maintaining large scale Big Data systems and Data Environments, used to ingest and process large data sets and provide actionable recommendations using data warehousing and business intelligence; use Linux OS; design, develop, test, and maintain AdTech software that extracts, transforms, and loads large volumes of data; develop software systems in an Agile development environment and on big data platforms, including Apache Spark, AWS, Flink, and Hadoop, using SQL, Presto, Scala, GoLang, Java, and Python; use AWS services including EC2, Lambda, S3, and Route 53; use Datadog and Grafana for monitoring; use Jenkins for CI/CD; develop Microservices API systems to support overall product development; store relational data in MySQL to support API and data processing applications; debug functional and performance issues on big data platforms and software modules; concurrently execute data processing software using multithreading; write code and scripts to extract MRM ad logs from FreeWheel Big Data platforms, and load MRM ad logs and campaign data; perform audience targeting; ingest audience, identity, and segment data; create dashboards and monitors on Datadog to ensure 24x7 availability of critical software deployments; build new software products and web frontend frameworks; analyze product specifications, write technical specs, create monitoring dashboards, develop test suites, design workflows, and setup database schemas and tables; interface with global engineering, operations, services, and business operations teams to execute proof of concepts and incorporate new requirements; improve system performance and ensure availability and scalability of services; provide production support for data processing systems running on AWS cloud and Snowflake; troubleshoot data processing problems running on distributed systems; and guide and mentor junior-level engineers. Position is eligible to work remotely within normal commuting distance of the worksite. Bachelor's degree, or foreign equivalent, in Computer Science, Engineering, or related technical field, and five (5) years of experience developing software in an Agile development environment using SQL, Presto, Scala, GoLang, Java, and Python on Big Data platforms including Apache Spark, AWS, Flink, and Hadoop; using Linux OS; using AWS services including EC2, Lambda, S3, and Route 53; using Datadog or Grafana for monitoring; using Jenkins for CI/CD; storing relational data in MySQL to support data processing applications; and debugging functional and performance issues on big data platforms and software modules. Disclaimer: This information has been designed to indicate the general nature and level of work performed by employees in this role. It is not designed to contain or be interpreted as a comprehensive inventory of all duties, responsibilities and qualifications. Skills Agile Environments, Apache Spark, Presto