Caliber Smart
Sr. Databricks Architect - Princeton, NJ (Onsite)
Caliber Smart, Princeton, New Jersey, us, 08543
Sr. Databricks Architect - Princeton, NJ (Onsite)
Caliber Smart is looking for a Sr. Databricks Architect to design & implement scalable data architectures using Delta Lake and Unity Catalog across major cloud platforms such as AWS, Azure, and GCP. The role requires strong expertise in data governance, streaming technologies, machine learning platforms, DevOps, and cost governance. Responsibilities
Contribute to the design and implementation of scalable data architectures, such as a Lakehouse, using Delta Lake and Unity Catalog. Manage and maintain the underlying data infrastructure on major cloud platforms. Implement data governance practices, including lineage, metadata management, and access controls integrating with third‑party products like Immuta and Protegrity. Adhere to software‑engineering best practices, including code reviews and CI/CD automation. Stay up to date with the latest trends and technologies in data engineering and the Databricks ecosystem. Design and build robust ETL/ELT workflows to ingest, transform, and load structured and unstructured data from various sources. Utilize Databricks features like Delta Live Tables and Databricks Workflows to orchestrate complex data processes. Optimize and tune Apache Spark jobs for performance and cost efficiency on large datasets. Implement and enforce data security, access control, and compliance policies in Databricks and Azure. Hands‑on experience with streaming technologies such as Kafka, Event Hubs, and Kinesis. Experience architecting machine‑learning platforms, advanced analytics workloads, and MLOps on Databricks. Expertise with enterprise security models, networking, and cost governance. Experience working with DevOps teams to establish deployment practices using Terraform or similar. Optimize Databricks performance (auto‑scaling, caching, delta optimization, job tuning, cost optimization). Prior experience leading enterprise Azure Databricks implementations. Qualifications
Mid‑Senior level experience. Full‑time Employment. Experience in data architecture, ETL/ELT design, Spark optimization, Databricks platform. Strong knowledge of Delta Lake, Unity Catalog, and data governance. Proficiency with cloud platforms (AWS, Azure, GCP). Hands‑on experience with Kafka, Event Hubs, Kinesis. Experience with Terraform or similar DevOps tools. Excellent communication and collaboration skills. Contact
Best Regards, Vishal vishal@smartcaliber.com www.smartcaliber.com Truth Lies in Heart
#J-18808-Ljbffr
Caliber Smart is looking for a Sr. Databricks Architect to design & implement scalable data architectures using Delta Lake and Unity Catalog across major cloud platforms such as AWS, Azure, and GCP. The role requires strong expertise in data governance, streaming technologies, machine learning platforms, DevOps, and cost governance. Responsibilities
Contribute to the design and implementation of scalable data architectures, such as a Lakehouse, using Delta Lake and Unity Catalog. Manage and maintain the underlying data infrastructure on major cloud platforms. Implement data governance practices, including lineage, metadata management, and access controls integrating with third‑party products like Immuta and Protegrity. Adhere to software‑engineering best practices, including code reviews and CI/CD automation. Stay up to date with the latest trends and technologies in data engineering and the Databricks ecosystem. Design and build robust ETL/ELT workflows to ingest, transform, and load structured and unstructured data from various sources. Utilize Databricks features like Delta Live Tables and Databricks Workflows to orchestrate complex data processes. Optimize and tune Apache Spark jobs for performance and cost efficiency on large datasets. Implement and enforce data security, access control, and compliance policies in Databricks and Azure. Hands‑on experience with streaming technologies such as Kafka, Event Hubs, and Kinesis. Experience architecting machine‑learning platforms, advanced analytics workloads, and MLOps on Databricks. Expertise with enterprise security models, networking, and cost governance. Experience working with DevOps teams to establish deployment practices using Terraform or similar. Optimize Databricks performance (auto‑scaling, caching, delta optimization, job tuning, cost optimization). Prior experience leading enterprise Azure Databricks implementations. Qualifications
Mid‑Senior level experience. Full‑time Employment. Experience in data architecture, ETL/ELT design, Spark optimization, Databricks platform. Strong knowledge of Delta Lake, Unity Catalog, and data governance. Proficiency with cloud platforms (AWS, Azure, GCP). Hands‑on experience with Kafka, Event Hubs, Kinesis. Experience with Terraform or similar DevOps tools. Excellent communication and collaboration skills. Contact
Best Regards, Vishal vishal@smartcaliber.com www.smartcaliber.com Truth Lies in Heart
#J-18808-Ljbffr