Genesis10
Sr Azure Data Engineer - 100% remote
Genesis10, Eden Prairie, Minnesota, United States, 55344
Genesis10 is currently seeking a Sr Azure Data Engineer for a 5 month contract role in Eden Prairie, MN. This role is 100% remote.
Compensation : $65.00 to $75.00/hour, W2
Job Summary
We are looking for a seasoned Azure Data Engineer with strong expertise in Azure Data Factory, Azure Databricks, and PySpark, who also brings hands-on experience in data migration and validation. The ideal candidate will have a proven track record of migrating Java-based data processing jobs to Azure Databricks using PySpark, and validating data between source and target systems using SQL and PySpark. This role requires strong hands-on experience with
Pyspark
and
Databricks , particularly in building data validation
test frameworks
to ensure accuracy from source to target. Additionally, prior experience in the healthcare domain is a key priority for us. Key Responsibilities
Design, develop, and maintain scalable data pipelines using Azure Data Factory and Azure Databricks. Migrate existing Java-based ETL jobs to PySpark on Azure Databricks. Implement standard data quality checks using SQL and PySpark (e.g., null checks, duplicates, referential integrity, record counts). Write efficient and optimized PySpark code for large-scale data processing. Collaborate with data analysts, architects, and business stakeholders to understand data requirements. Perform unit testing and validation of data transformations and ensure data quality. Develop and maintain automated validation frameworks for continuous data quality assurance. Monitor and troubleshoot data pipelines and workflows in production environments. Document technical solutions, migration strategies, and validation procedures. Required Skills & Qualifications
Minimum 6 years of experience in data engineering or related roles. Strong hands-on experience with Azure Data Factory, Azure Databricks, and PySpark. Proven experience in migrating Java-based data jobs to PySpark on Azure Databricks. Solid understanding of SQL and experience in data validation and testing. Familiarity with standard data quality checks and validation frameworks. Experience working in cloud-based environments, preferably Azure. Strong problem-solving skills and ability to work independently or in a team. Excellent communication and documentation skills. Preferred Qualifications
Experience with CI/CD pipelines and DevOps practices in data engineering. Experience working with Talend Studio. Knowledge of Delta Lake, Spark optimization techniques, and data lake architectures. Exposure to Agile methodologies and project management tools. We have access to additional contract, contract-to-hire, and direct hire positions with various rate ranges. If you have the qualifications described and are interested in this exciting opportunity, apply today! Genesis10 is an Equal Opportunity Employer. Candidates will receive consideration without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.
#J-18808-Ljbffr
We are looking for a seasoned Azure Data Engineer with strong expertise in Azure Data Factory, Azure Databricks, and PySpark, who also brings hands-on experience in data migration and validation. The ideal candidate will have a proven track record of migrating Java-based data processing jobs to Azure Databricks using PySpark, and validating data between source and target systems using SQL and PySpark. This role requires strong hands-on experience with
Pyspark
and
Databricks , particularly in building data validation
test frameworks
to ensure accuracy from source to target. Additionally, prior experience in the healthcare domain is a key priority for us. Key Responsibilities
Design, develop, and maintain scalable data pipelines using Azure Data Factory and Azure Databricks. Migrate existing Java-based ETL jobs to PySpark on Azure Databricks. Implement standard data quality checks using SQL and PySpark (e.g., null checks, duplicates, referential integrity, record counts). Write efficient and optimized PySpark code for large-scale data processing. Collaborate with data analysts, architects, and business stakeholders to understand data requirements. Perform unit testing and validation of data transformations and ensure data quality. Develop and maintain automated validation frameworks for continuous data quality assurance. Monitor and troubleshoot data pipelines and workflows in production environments. Document technical solutions, migration strategies, and validation procedures. Required Skills & Qualifications
Minimum 6 years of experience in data engineering or related roles. Strong hands-on experience with Azure Data Factory, Azure Databricks, and PySpark. Proven experience in migrating Java-based data jobs to PySpark on Azure Databricks. Solid understanding of SQL and experience in data validation and testing. Familiarity with standard data quality checks and validation frameworks. Experience working in cloud-based environments, preferably Azure. Strong problem-solving skills and ability to work independently or in a team. Excellent communication and documentation skills. Preferred Qualifications
Experience with CI/CD pipelines and DevOps practices in data engineering. Experience working with Talend Studio. Knowledge of Delta Lake, Spark optimization techniques, and data lake architectures. Exposure to Agile methodologies and project management tools. We have access to additional contract, contract-to-hire, and direct hire positions with various rate ranges. If you have the qualifications described and are interested in this exciting opportunity, apply today! Genesis10 is an Equal Opportunity Employer. Candidates will receive consideration without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.
#J-18808-Ljbffr