4P Consulting Inc
Senior Data Engineer
Location: Atlanta, GA
Client- Southern Company
Contract- 1 Year Experience Level:
10+ Years Position Summary:
We are seeking an experienced
Senior Data Engineer
with a strong background in designing and developing robust, scalable, and high-performance data infrastructure. The ideal candidate will have 10+ years of experience in manipulating data, working with both relational and NoSQL systems, and building modern data pipelines across data lakes and warehouses. This role demands a deep technical skillset in SQL, data modeling, and data pipeline orchestration tools like
Apache Spark, Hive, and Airflow . Key Responsibilities:
Design, develop, and maintain scalable
ETL/ELT pipelines
to process and integrate data from disparate sources.
Work with
structured and unstructured data
to build clean, efficient, and machine-readable datasets.
Normalize databases and ensure that data structures align with application and business requirements.
Collaborate with business analysts, data scientists, and software engineers to deliver data solutions aligned with company objectives.
Design and implement
data models
that support reporting, analytics, and machine learning use cases.
Migrate and optimize workloads in
cloud-based environments
(e.g., Azure, AWS, GCP).
Monitor and optimize data pipeline performance, data quality, and reliability.
Implement
data governance
best practices and ensure compliance with security and privacy regulations.
Technical Skills & Expertise:
Expert-level proficiency in
SQL , including writing complex queries and optimizing performance.
Deep understanding of
data modeling techniques
for both OLTP and OLAP systems.
Hands-on experience with
big data frameworks : Apache Spark ,
Hive ,
Airflow ,
Databricks
Experience with modern
cloud data platforms : Azure Data Lake ,
Azure Synapse ,
AWS Redshift , or
Google BigQuery
Strong experience with
data orchestration ,
scheduling , and
automation tools .
Familiarity with
Python ,
Scala , or
Java
for data processing and scripting tasks.
Knowledge of
NoSQL databases
(e.g., MongoDB, Cassandra) and
streaming tools
(e.g., Kafka) is a plus.
Experience with
data security, lineage, and cataloging tools
(e.g., Unity Catalog, Collibra).
Preferred Qualifications:
Bachelors or Masters degree in
Computer Science ,
Data Engineering ,
Information Systems , or a related field.
Microsoft
Azure Data Engineer Associate
or equivalent certifications preferred.
Experience in
data governance
processes and supporting enterprise
analytics platforms .
Knowledge of
containerization
(e.g., Docker) and
CI/CD pipelines
in DevOps environments.
#J-18808-Ljbffr
Location: Atlanta, GA
Client- Southern Company
Contract- 1 Year Experience Level:
10+ Years Position Summary:
We are seeking an experienced
Senior Data Engineer
with a strong background in designing and developing robust, scalable, and high-performance data infrastructure. The ideal candidate will have 10+ years of experience in manipulating data, working with both relational and NoSQL systems, and building modern data pipelines across data lakes and warehouses. This role demands a deep technical skillset in SQL, data modeling, and data pipeline orchestration tools like
Apache Spark, Hive, and Airflow . Key Responsibilities:
Design, develop, and maintain scalable
ETL/ELT pipelines
to process and integrate data from disparate sources.
Work with
structured and unstructured data
to build clean, efficient, and machine-readable datasets.
Normalize databases and ensure that data structures align with application and business requirements.
Collaborate with business analysts, data scientists, and software engineers to deliver data solutions aligned with company objectives.
Design and implement
data models
that support reporting, analytics, and machine learning use cases.
Migrate and optimize workloads in
cloud-based environments
(e.g., Azure, AWS, GCP).
Monitor and optimize data pipeline performance, data quality, and reliability.
Implement
data governance
best practices and ensure compliance with security and privacy regulations.
Technical Skills & Expertise:
Expert-level proficiency in
SQL , including writing complex queries and optimizing performance.
Deep understanding of
data modeling techniques
for both OLTP and OLAP systems.
Hands-on experience with
big data frameworks : Apache Spark ,
Hive ,
Airflow ,
Databricks
Experience with modern
cloud data platforms : Azure Data Lake ,
Azure Synapse ,
AWS Redshift , or
Google BigQuery
Strong experience with
data orchestration ,
scheduling , and
automation tools .
Familiarity with
Python ,
Scala , or
Java
for data processing and scripting tasks.
Knowledge of
NoSQL databases
(e.g., MongoDB, Cassandra) and
streaming tools
(e.g., Kafka) is a plus.
Experience with
data security, lineage, and cataloging tools
(e.g., Unity Catalog, Collibra).
Preferred Qualifications:
Bachelors or Masters degree in
Computer Science ,
Data Engineering ,
Information Systems , or a related field.
Microsoft
Azure Data Engineer Associate
or equivalent certifications preferred.
Experience in
data governance
processes and supporting enterprise
analytics platforms .
Knowledge of
containerization
(e.g., Docker) and
CI/CD pipelines
in DevOps environments.
#J-18808-Ljbffr