Jobs via Dice
Data Engineer
6 Month(s) Contract to hire
Raleigh, NC
Key Responsibilities
Design, build, and maintain scalable ETL data pipeline using ADLS Gen2, Azure Blob storage, Azure Databricks, Python, Azure SQL Database and Azure Cosmos DB.
Perform deep database performance tuning, query optimization, and indexing strategies across Azure SQL Database and Cosmos DB.
Lead database migration initiatives, including strategic migration from Azure SQL Database and Cosmos DB to PostgreSQL.
Implement efficient data models and manage data storage solutions using ADLS Gen2 and real‑time data streaming using ADF, Spark and Confluent Kafka.
Develop and maintain data quality checks, monitoring, and alerting systems. Ensure data solutions comply with regulatory and security standards (e.g., PCI DSS).
Optimize batch processing workflows for intra‑day and end‑of‑day banking operations.
Required Qualifications
8+ Years of experience in data engineering with strong database expertise.
At least 5 years of experience implementing an operational data store (ODS) for the retail banking space, capturing multiple cross‑system transactions and delivering high throughput and low latency solutions using RDBMS or NoSQL technologies.
Hands‑on experience implementing solutions with ADLS Gen2, Azure Blob storage, Azure Databricks, Azure SQL Database and Azure Cosmos DB.
Experience writing queries using the normalized data model for the retail banking space. Expert‑level skills in database performance tuning, query optimization, and execution plan analysis.
Proven experience with large‑scale database migrations (Azure SQL Database to PostgreSQL preferred).
Strong programming skills in Python for data processing and automation.
Experience with data modeling, ETL/ELT processes, and data warehouse concepts.
Excellent problem‑solving skills with the ability to diagnose and resolve complex data pipeline issues.
Seniority level Mid‑Senior level
Employment type Full‑time
Job function Information Technology
Industries Software Development
Salary:
$94,550 – $151,170
Thank You Kanishk Pratap
#J-18808-Ljbffr
6 Month(s) Contract to hire
Raleigh, NC
Key Responsibilities
Design, build, and maintain scalable ETL data pipeline using ADLS Gen2, Azure Blob storage, Azure Databricks, Python, Azure SQL Database and Azure Cosmos DB.
Perform deep database performance tuning, query optimization, and indexing strategies across Azure SQL Database and Cosmos DB.
Lead database migration initiatives, including strategic migration from Azure SQL Database and Cosmos DB to PostgreSQL.
Implement efficient data models and manage data storage solutions using ADLS Gen2 and real‑time data streaming using ADF, Spark and Confluent Kafka.
Develop and maintain data quality checks, monitoring, and alerting systems. Ensure data solutions comply with regulatory and security standards (e.g., PCI DSS).
Optimize batch processing workflows for intra‑day and end‑of‑day banking operations.
Required Qualifications
8+ Years of experience in data engineering with strong database expertise.
At least 5 years of experience implementing an operational data store (ODS) for the retail banking space, capturing multiple cross‑system transactions and delivering high throughput and low latency solutions using RDBMS or NoSQL technologies.
Hands‑on experience implementing solutions with ADLS Gen2, Azure Blob storage, Azure Databricks, Azure SQL Database and Azure Cosmos DB.
Experience writing queries using the normalized data model for the retail banking space. Expert‑level skills in database performance tuning, query optimization, and execution plan analysis.
Proven experience with large‑scale database migrations (Azure SQL Database to PostgreSQL preferred).
Strong programming skills in Python for data processing and automation.
Experience with data modeling, ETL/ELT processes, and data warehouse concepts.
Excellent problem‑solving skills with the ability to diagnose and resolve complex data pipeline issues.
Seniority level Mid‑Senior level
Employment type Full‑time
Job function Information Technology
Industries Software Development
Salary:
$94,550 – $151,170
Thank You Kanishk Pratap
#J-18808-Ljbffr