Daten
Overview
We are seeking an experienced
Big Data Lead
to design, implement, and optimize large-scale data engineering solutions on modern cloud and distributed computing platforms. The role requires strong technical expertise in Big Data frameworks, advanced SQL, data integration, and cloud-native services. The ideal candidate will have proven experience in leading data projects, building scalable data pipelines, and delivering actionable insights through analytics and visualization platforms. Education Requirements Bachelors degree in computer science or a related technical field. Minimum 5 years of relevant experience in Big Data Engineering/Data Analysis.
Experience Requirements
Strong proficiency in
Python, SQL, and Apache Spark
(Core Spark, Spark Streaming, DataFrame API, DataSet API, RDD, Spark SQL). Hands-on experience with
AWS services
including EMR, Glue (serverless architecture), S3, Athena, IAM, Lambda, and CloudWatch. Expertise in
Hadoop
and distributed computing frameworks. Advanced SQL development with Hive/Impala including performance tuning. Experience with
ElasticSearch (OpenSearch)
and
Kibana dashboards . Familiarity with resource management frameworks such as Yarn or Mesos. Knowledge of external job schedulers like Autosys, AWS Data Pipeline, or Airflow. Experience in
HBase
and other Key/Value data stores.
Role & Responsibilities
Design, develop, and maintain scalable data pipelines across Big Data platforms. Perform physical data modeling and table design for high-volume data systems. Build and manage data catalogs and dictionaries to maintain data integrity. Deliver solutions using AWS services (EMR, Glue, Lambda, S3, Athena, Redshift, Snowflake). Optimize big data workloads leveraging Hadoop, Spark, and cloud-native frameworks.
Data Integration & ETL
Clean, transform, and blend data from multiple sources using ETL tools. Implement automation for data ingestion, transformation, and monitoring processes. Deliver self-service analytics, KPIs, and dashboards for business teams. Develop Tableau dashboards, including calculated fields and interactive reports. Support advanced analytics by integrating ElasticSearch/OpenSearch with Kibana. Work closely with cross-functional teams to define business requirements and translate them into technical data solutions. Lead and mentor a team of engineers to deliver high-quality data-driven outcomes. Ensure effective communication between business stakeholders, data engineering, and analytics teams.
Performance & Optimization
Conduct SQL and query optimization for high-performance analytics. Monitor data pipelines, troubleshoot performance bottlenecks, and implement scalable solutions. Perform root cause analysis and ensure reliability of data platforms.
#J-18808-Ljbffr
We are seeking an experienced
Big Data Lead
to design, implement, and optimize large-scale data engineering solutions on modern cloud and distributed computing platforms. The role requires strong technical expertise in Big Data frameworks, advanced SQL, data integration, and cloud-native services. The ideal candidate will have proven experience in leading data projects, building scalable data pipelines, and delivering actionable insights through analytics and visualization platforms. Education Requirements Bachelors degree in computer science or a related technical field. Minimum 5 years of relevant experience in Big Data Engineering/Data Analysis.
Experience Requirements
Strong proficiency in
Python, SQL, and Apache Spark
(Core Spark, Spark Streaming, DataFrame API, DataSet API, RDD, Spark SQL). Hands-on experience with
AWS services
including EMR, Glue (serverless architecture), S3, Athena, IAM, Lambda, and CloudWatch. Expertise in
Hadoop
and distributed computing frameworks. Advanced SQL development with Hive/Impala including performance tuning. Experience with
ElasticSearch (OpenSearch)
and
Kibana dashboards . Familiarity with resource management frameworks such as Yarn or Mesos. Knowledge of external job schedulers like Autosys, AWS Data Pipeline, or Airflow. Experience in
HBase
and other Key/Value data stores.
Role & Responsibilities
Design, develop, and maintain scalable data pipelines across Big Data platforms. Perform physical data modeling and table design for high-volume data systems. Build and manage data catalogs and dictionaries to maintain data integrity. Deliver solutions using AWS services (EMR, Glue, Lambda, S3, Athena, Redshift, Snowflake). Optimize big data workloads leveraging Hadoop, Spark, and cloud-native frameworks.
Data Integration & ETL
Clean, transform, and blend data from multiple sources using ETL tools. Implement automation for data ingestion, transformation, and monitoring processes. Deliver self-service analytics, KPIs, and dashboards for business teams. Develop Tableau dashboards, including calculated fields and interactive reports. Support advanced analytics by integrating ElasticSearch/OpenSearch with Kibana. Work closely with cross-functional teams to define business requirements and translate them into technical data solutions. Lead and mentor a team of engineers to deliver high-quality data-driven outcomes. Ensure effective communication between business stakeholders, data engineering, and analytics teams.
Performance & Optimization
Conduct SQL and query optimization for high-performance analytics. Monitor data pipelines, troubleshoot performance bottlenecks, and implement scalable solutions. Perform root cause analysis and ensure reliability of data platforms.
#J-18808-Ljbffr