Caliber Smart
Data Architect/ Databricks Architect- Princeton, NJ (Onsite)
Caliber Smart, Princeton, New Jersey, us, 08543
Data Architect/ Databricks Architect- Princeton, NJ (Onsite)
Join to apply for the Data Architect/ Databricks Architect- Princeton, NJ (Onsite) role at
Caliber Smart .
Role Name:
Databricks Architect Location:
Princeton, NJ (Onsite) Duration:
12 Months Job ID:
CGEMJP00324803
Job Description:
Contribute to the design and implementation of scalable data architectures, such as a Lakehouse, using Delta Lake and Unity Catalog.
Manage and maintain the underlying data infrastructure, which typically exists on a major cloud platform like AWS, Azure, or GCP.
Implement data governance practices, including data lineage, metadata management, and access controls integrating with other 3rd party products like Immuta, Protegrity for tokenization etc.
Adhere to software engineering best practices, including participating in code reviews and CI/CD (Continuous Integration/Continuous Deployment) automation.
Stay up to date with the latest trends and technologies in data engineering and the Databricks ecosystem.
Design and build robust ETL (Extract, Transform, Load) and ELT workflows to ingest, transform, and load structured and unstructured data from various sources.
Utilize Databricks features like Delta Live Tables and Databricks Workflows to orchestrate and manage complex data processes.
Optimize and tune Apache Spark jobs for performance and cost efficiency on large datasets.
Implement and enforce data security, access control, and compliance policies in Databricks and Azure.
Hands on experience working on streaming technologies (Kafka, Event Hubs, Kinesis).
Experience architecting machine learning platforms, advanced analytics workloads and develop and deployment of MLOPS on Databricks.
Expertise with enterprise security models, networking, and cost governance.
Should have experience working with DevOps teams to establish deployment practices using Terraform or similar.
Optimize Databricks performance (auto-scaling, caching, delta optimization, job tuning, cost optimization).
Prior experience in leading enterprise Azure Databricks implementations.
Best Regards, Vishal vishal@smartcaliber.com www.smartcaliber.com Truth Lies in Heart
#J-18808-Ljbffr
Caliber Smart .
Role Name:
Databricks Architect Location:
Princeton, NJ (Onsite) Duration:
12 Months Job ID:
CGEMJP00324803
Job Description:
Contribute to the design and implementation of scalable data architectures, such as a Lakehouse, using Delta Lake and Unity Catalog.
Manage and maintain the underlying data infrastructure, which typically exists on a major cloud platform like AWS, Azure, or GCP.
Implement data governance practices, including data lineage, metadata management, and access controls integrating with other 3rd party products like Immuta, Protegrity for tokenization etc.
Adhere to software engineering best practices, including participating in code reviews and CI/CD (Continuous Integration/Continuous Deployment) automation.
Stay up to date with the latest trends and technologies in data engineering and the Databricks ecosystem.
Design and build robust ETL (Extract, Transform, Load) and ELT workflows to ingest, transform, and load structured and unstructured data from various sources.
Utilize Databricks features like Delta Live Tables and Databricks Workflows to orchestrate and manage complex data processes.
Optimize and tune Apache Spark jobs for performance and cost efficiency on large datasets.
Implement and enforce data security, access control, and compliance policies in Databricks and Azure.
Hands on experience working on streaming technologies (Kafka, Event Hubs, Kinesis).
Experience architecting machine learning platforms, advanced analytics workloads and develop and deployment of MLOPS on Databricks.
Expertise with enterprise security models, networking, and cost governance.
Should have experience working with DevOps teams to establish deployment practices using Terraform or similar.
Optimize Databricks performance (auto-scaling, caching, delta optimization, job tuning, cost optimization).
Prior experience in leading enterprise Azure Databricks implementations.
Best Regards, Vishal vishal@smartcaliber.com www.smartcaliber.com Truth Lies in Heart
#J-18808-Ljbffr