Bizmetric
About The Role
We are seeking an experienced Senior Databricks Engineer to design, develop, optimize, and manage large-scale data pipelines and analytics solutions using Databricks, Spark, and modern cloud platforms (Azure/AWS/GCP). The ideal candidate has deep expertise in distributed data processing, Delta Lake architecture, and end-to-end data engineering best practices. You will work with cross-functional teams—including product, analytics, and delivery—to deliver scalable data solutions for enterprise clients.
Job Title Senior Databricks Engineer
Location Onsite – Sugarland, Tx.
Employment Type Full-Time
Seniority level Mid-Senior level
Job Function Engineering and Information Technology
Industries IT Services and IT Consulting
Key Responsibilities Databricks & Spark Engineering
Design, develop, and optimize Spark/Databricks ETL/ELT pipelines at scale.
Implement efficient Delta Lake architectures (bronze/silver/gold layers).
Write high-performance PySpark/SQL code following best practices.
Tune clusters, jobs, and workflows for performance and cost efficiency.
Data Architecture & Modeling
Develop data models and schemas to support analytics, reporting, and ML workloads.
Implement medallion architecture following industry and Databricks standards.
Design scalable and secure storage patterns using cloud-native services.
Cloud Engineering & Integration
Work with cloud platforms (Azure preferred, AWS or GCP acceptable).
Implement CI/CD pipelines for Databricks using Git, Azure DevOps, or GitHub Actions.
Integrate Databricks with third-party systems, ADF, Event Hub, Kafka, APIs, etc.
Configure workflows in Databricks (Jobs, Workflows, Unity Catalog).
Data Quality, Governance & Security
Implement data validation, observability, and monitoring frameworks.
Apply best practices in data governance, lineage, and cataloging.
Enforce security guidelines using RBAC, Unity Catalog, and encryption policies.
Collaboration & Leadership
Lead technical solutioning for client projects and POCs.
Provide mentorship to junior engineers.
Collaborate closely with business analysts, product managers, and stakeholders.
Prepare architectural documentation, standards, and reusable accelerators.
Troubleshooting & Optimization
Investigate data issues, performance bottlenecks, and operational failures.
Drive continuous improvement across data engineering processes.
Ensure reliable and timely data delivery for downstream applications.
#J-18808-Ljbffr
Job Title Senior Databricks Engineer
Location Onsite – Sugarland, Tx.
Employment Type Full-Time
Seniority level Mid-Senior level
Job Function Engineering and Information Technology
Industries IT Services and IT Consulting
Key Responsibilities Databricks & Spark Engineering
Design, develop, and optimize Spark/Databricks ETL/ELT pipelines at scale.
Implement efficient Delta Lake architectures (bronze/silver/gold layers).
Write high-performance PySpark/SQL code following best practices.
Tune clusters, jobs, and workflows for performance and cost efficiency.
Data Architecture & Modeling
Develop data models and schemas to support analytics, reporting, and ML workloads.
Implement medallion architecture following industry and Databricks standards.
Design scalable and secure storage patterns using cloud-native services.
Cloud Engineering & Integration
Work with cloud platforms (Azure preferred, AWS or GCP acceptable).
Implement CI/CD pipelines for Databricks using Git, Azure DevOps, or GitHub Actions.
Integrate Databricks with third-party systems, ADF, Event Hub, Kafka, APIs, etc.
Configure workflows in Databricks (Jobs, Workflows, Unity Catalog).
Data Quality, Governance & Security
Implement data validation, observability, and monitoring frameworks.
Apply best practices in data governance, lineage, and cataloging.
Enforce security guidelines using RBAC, Unity Catalog, and encryption policies.
Collaboration & Leadership
Lead technical solutioning for client projects and POCs.
Provide mentorship to junior engineers.
Collaborate closely with business analysts, product managers, and stakeholders.
Prepare architectural documentation, standards, and reusable accelerators.
Troubleshooting & Optimization
Investigate data issues, performance bottlenecks, and operational failures.
Drive continuous improvement across data engineering processes.
Ensure reliable and timely data delivery for downstream applications.
#J-18808-Ljbffr