Fractal
Base pay range
$110,000.00/yr - $145,000.00/yr
Role Overview We are seeking a skilled Lead Data Engineer to support the migration of on‑premise SQL Server databases to the Azure Databricks Lakehouse Platform. This role involves designing and implementing parameter‑driven ETL frameworks, migrating SQL stored procedures, views, and business logic to Databricks Unity Catalog stored procedures, and supporting data governance strategies. The ideal candidate will have proven experience with Databricks, Unity Catalog, Delta Lake, and ADF, and will work closely with the team to deliver scalable, efficient, and secure data migration solutions.
Key Responsibilities
Design and implement data migration pipelines from on‑prem SQL Server to Azure Databricks Lakehouse.
Build parameter‑driven/config‑based ETL mappings for reusable and dynamic pipelines.
Migrate SQL stored procedures, views, and business logic from SQL Server to Databricks Unity Catalog stored procedures using Databricks SQL or PySpark.
Configure and manage Databricks Unity Catalog for governance, lineage, access control, and security.
Develop and optimise Delta Lake tables following the Medallion architecture (Bronze, Silver, Gold).
Orchestrate ETL workflows via Azure Data Factory (ADF) and Databricks Workflows.
Implement data access controls, account‑level permissions, and object ownership via Unity Catalog.
Collaborate with architects, developers, and business stakeholders to ensure data quality, lineage, and governance.
Troubleshoot and optimise ETL pipelines for performance, scalability, and cost efficiency.
Support CI/CD integration using Databricks Repos, Git, and Azure DevOps.
Required Skills
10+ years of experience in data engineering, including cloud migration projects.
Hands‑on expertise in Azure Databricks, ADF, ADLS Gen2, and Databricks Unity Catalog.
Very good knowledge of SQL DB stored procs, SSIS, SSRS.
Proven experience in migrating SQL stored procedures, views, and business logic to Databricks Unity Catalog stored procedures.
Strong skills in Databricks SQL, PySpark, and Python for data transformation.
Deep understanding of Delta Lake, Medallion architecture, and data modelling.
Knowledge of data governance, lineage tracking, and security policies.
Experience with CI/CD automation, Git, and Azure DevOps.
Visa Sponsorship We are currently unable to sponsor visas. Only visa‑independent candidates will be considered.
Pay The wage range for this role takes into account the wide range of factors that are considered in making compensation decisions including but not limited to skill sets; experience and training; licensure and certifications; and other business and organisational needs. The disclosed range estimate has not been adjusted for the applicable geographic differential associated with the location at which the position may be filled. At Fractal, it is not typical for an individual to be hired at or near the top of the range for their role and compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current range is
$110,000 – $145,000 . In addition, you may be eligible for a discretionary bonus for the current performance period.
Benefits As a full‑time employee of the company or as an hourly employee working more than 30 hours per week, you will be eligible to participate in the health, dental, vision, life insurance, and disability plans in accordance with the plan documents, which may be amended from time to time. You will be eligible for benefits on the first day of employment with the Company. In addition, you are eligible to participate in the Company 401(k) Plan after 30 days of employment, in accordance with the applicable plan terms. The Company provides for 11 paid holidays and 12 weeks of parental leave. We also follow a “free time” PTO policy, allowing you the flexibility to take time needed for either sick time or vacation.
Equal Employment Opportunity Fractal provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.
#J-18808-Ljbffr
Role Overview We are seeking a skilled Lead Data Engineer to support the migration of on‑premise SQL Server databases to the Azure Databricks Lakehouse Platform. This role involves designing and implementing parameter‑driven ETL frameworks, migrating SQL stored procedures, views, and business logic to Databricks Unity Catalog stored procedures, and supporting data governance strategies. The ideal candidate will have proven experience with Databricks, Unity Catalog, Delta Lake, and ADF, and will work closely with the team to deliver scalable, efficient, and secure data migration solutions.
Key Responsibilities
Design and implement data migration pipelines from on‑prem SQL Server to Azure Databricks Lakehouse.
Build parameter‑driven/config‑based ETL mappings for reusable and dynamic pipelines.
Migrate SQL stored procedures, views, and business logic from SQL Server to Databricks Unity Catalog stored procedures using Databricks SQL or PySpark.
Configure and manage Databricks Unity Catalog for governance, lineage, access control, and security.
Develop and optimise Delta Lake tables following the Medallion architecture (Bronze, Silver, Gold).
Orchestrate ETL workflows via Azure Data Factory (ADF) and Databricks Workflows.
Implement data access controls, account‑level permissions, and object ownership via Unity Catalog.
Collaborate with architects, developers, and business stakeholders to ensure data quality, lineage, and governance.
Troubleshoot and optimise ETL pipelines for performance, scalability, and cost efficiency.
Support CI/CD integration using Databricks Repos, Git, and Azure DevOps.
Required Skills
10+ years of experience in data engineering, including cloud migration projects.
Hands‑on expertise in Azure Databricks, ADF, ADLS Gen2, and Databricks Unity Catalog.
Very good knowledge of SQL DB stored procs, SSIS, SSRS.
Proven experience in migrating SQL stored procedures, views, and business logic to Databricks Unity Catalog stored procedures.
Strong skills in Databricks SQL, PySpark, and Python for data transformation.
Deep understanding of Delta Lake, Medallion architecture, and data modelling.
Knowledge of data governance, lineage tracking, and security policies.
Experience with CI/CD automation, Git, and Azure DevOps.
Visa Sponsorship We are currently unable to sponsor visas. Only visa‑independent candidates will be considered.
Pay The wage range for this role takes into account the wide range of factors that are considered in making compensation decisions including but not limited to skill sets; experience and training; licensure and certifications; and other business and organisational needs. The disclosed range estimate has not been adjusted for the applicable geographic differential associated with the location at which the position may be filled. At Fractal, it is not typical for an individual to be hired at or near the top of the range for their role and compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current range is
$110,000 – $145,000 . In addition, you may be eligible for a discretionary bonus for the current performance period.
Benefits As a full‑time employee of the company or as an hourly employee working more than 30 hours per week, you will be eligible to participate in the health, dental, vision, life insurance, and disability plans in accordance with the plan documents, which may be amended from time to time. You will be eligible for benefits on the first day of employment with the Company. In addition, you are eligible to participate in the Company 401(k) Plan after 30 days of employment, in accordance with the applicable plan terms. The Company provides for 11 paid holidays and 12 weeks of parental leave. We also follow a “free time” PTO policy, allowing you the flexibility to take time needed for either sick time or vacation.
Equal Employment Opportunity Fractal provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.
#J-18808-Ljbffr