ZipRecruiter
Senior Data Engineer - Azure Databricks
ZipRecruiter, Nashville, Tennessee, United States, 37247
Overview
We are seeking an accomplished
Senior Data Engineer
with deep expertise in
Azure Databricks, advanced data engineering, and healthcare data science . This role is critical to our mission of leveraging data to transform healthcare outcomes. The ideal candidate will have a proven ability to design and scale enterprise-grade data pipelines, optimize workflows in the cloud, and deliver predictive insights that directly impact patient care and operational efficiency. As part of our data engineering team, you will collaborate with data scientists, analysts, and software engineers to deliver secure, high-performance solutions that adhere to healthcare standards and compliance requirements. Key Responsibilities
Design, build, and optimize
scalable ETL/ELT pipelines in
Azure
for diverse healthcare data, including EHR, claims, IoT, and unstructured sources. Implement advanced
data storage and processing solutions
leveraging Azure Data Lake, Synapse Analytics, Databricks, and Azure SQL. Partner with data scientists to
develop, deploy, and operationalize machine learning models
and predictive analytics solutions supporting healthcare use cases. Ensure
data quality, governance, security, and compliance
with HIPAA, PHI/PII, and other healthcare regulations. Automate workflows using
Azure Data Factory, Databricks jobs, and CI/CD pipelines
to enhance reliability and reduce manual effort. Monitor, troubleshoot, and fine-tune production pipelines for
scalability, cost efficiency, and high availability . Collaborate cross-functionally to translate business needs into robust technical solutions, aligning data engineering practices with healthcare standards (FHIR, HL7). Contribute to the evolution of our
data architecture strategy , incorporating best practices for cloud optimization, data governance, and emerging healthcare data trends. Required Skills & Qualifications
8+ years of experience as a
Data Engineer , with extensive work in
Azure cloud services . Strong proficiency in
SQL, Python, PySpark, and Databricks
for data engineering and advanced analytics. Hands-on expertise with
Azure Data Factory, Azure Synapse, Azure Data Lake, and Power BI . Experience designing and implementing
machine learning solutions
in a healthcare environment. Working knowledge of
FHIR, HL7, and other healthcare interoperability standards . Familiarity with
DevOps practices and CI/CD pipelines
for automating data workflows. Strong analytical and problem-solving skills; proven ability to deliver in fast-paced, agile environments. Qualifications
Experience with
Big Data ecosystems
(Kafka, Hadoop, etc.) for real-time and batch data processing. Knowledge of
data governance frameworks
and compliance regulations in healthcare. Microsoft certifications such as
Azure Data Engineer Associate
or Databricks certifications. Understanding of
cloud cost optimization strategies
for large-scale data workloads. Why Join Us?
At Kanini, you will have the opportunity to: Shape the
future of healthcare data
through innovative cloud and data science solutions. Work with cutting-edge technologies while ensuring compliance and security in one of the most critical industries. Collaborate with a passionate, mission-driven team that values innovation, agility, and excellence.
#J-18808-Ljbffr
We are seeking an accomplished
Senior Data Engineer
with deep expertise in
Azure Databricks, advanced data engineering, and healthcare data science . This role is critical to our mission of leveraging data to transform healthcare outcomes. The ideal candidate will have a proven ability to design and scale enterprise-grade data pipelines, optimize workflows in the cloud, and deliver predictive insights that directly impact patient care and operational efficiency. As part of our data engineering team, you will collaborate with data scientists, analysts, and software engineers to deliver secure, high-performance solutions that adhere to healthcare standards and compliance requirements. Key Responsibilities
Design, build, and optimize
scalable ETL/ELT pipelines in
Azure
for diverse healthcare data, including EHR, claims, IoT, and unstructured sources. Implement advanced
data storage and processing solutions
leveraging Azure Data Lake, Synapse Analytics, Databricks, and Azure SQL. Partner with data scientists to
develop, deploy, and operationalize machine learning models
and predictive analytics solutions supporting healthcare use cases. Ensure
data quality, governance, security, and compliance
with HIPAA, PHI/PII, and other healthcare regulations. Automate workflows using
Azure Data Factory, Databricks jobs, and CI/CD pipelines
to enhance reliability and reduce manual effort. Monitor, troubleshoot, and fine-tune production pipelines for
scalability, cost efficiency, and high availability . Collaborate cross-functionally to translate business needs into robust technical solutions, aligning data engineering practices with healthcare standards (FHIR, HL7). Contribute to the evolution of our
data architecture strategy , incorporating best practices for cloud optimization, data governance, and emerging healthcare data trends. Required Skills & Qualifications
8+ years of experience as a
Data Engineer , with extensive work in
Azure cloud services . Strong proficiency in
SQL, Python, PySpark, and Databricks
for data engineering and advanced analytics. Hands-on expertise with
Azure Data Factory, Azure Synapse, Azure Data Lake, and Power BI . Experience designing and implementing
machine learning solutions
in a healthcare environment. Working knowledge of
FHIR, HL7, and other healthcare interoperability standards . Familiarity with
DevOps practices and CI/CD pipelines
for automating data workflows. Strong analytical and problem-solving skills; proven ability to deliver in fast-paced, agile environments. Qualifications
Experience with
Big Data ecosystems
(Kafka, Hadoop, etc.) for real-time and batch data processing. Knowledge of
data governance frameworks
and compliance regulations in healthcare. Microsoft certifications such as
Azure Data Engineer Associate
or Databricks certifications. Understanding of
cloud cost optimization strategies
for large-scale data workloads. Why Join Us?
At Kanini, you will have the opportunity to: Shape the
future of healthcare data
through innovative cloud and data science solutions. Work with cutting-edge technologies while ensuring compliance and security in one of the most critical industries. Collaborate with a passionate, mission-driven team that values innovation, agility, and excellence.
#J-18808-Ljbffr