Diverse Lynx
Role: Data Bricks Architect
Location: New York, NY (Onsite)
Contract
Job Description: We are looking for an experienced Databricks Architect with deep expertise in Lakehouse and Medallion Architecture (Bronze, Silver, Gold layers) to lead the design and implementation of scalable, high-performance data solutions on the Databricks platform. The ideal candidate will have a strong background in big data architecture, data governance, and cloud-native data engineering using technologies like Delta Lake, Spark, and Apache Airflow.
Key Responsibilities : Design, architect, and implement end-to-end data pipelines using Databricks Lakehouse and Medallion architecture (Bronze, Silver, Gold layers). Lead development teams in building scalable and reusable frameworks for ingestion, transformation, and analytics. Define data modeling strategies and partitioning schemes to optimize performance and cost. Ensure data governance, quality, and security standards are incorporated throughout the architecture. Work closely with business stakeholders, data scientists, and analysts to align technical architecture with business objectives. Set up and manage CI/CD pipelines, orchestrators (e.g., Airflow, Databricks Workflows), and Delta Live Tables. Optimize Spark workloads and recommend performance tuning best practices for large-scale data processing. Act as a subject matter expert for Databricks, Delta Lake, Unity Catalog, and cloud-native data platforms (e.g., AWS, Azure, GCP). Conduct design reviews, code reviews, and technical workshops to mentor engineering teams. Required Skills and Qualifications:
Proven expertise with Databricks, Apache Spark, and Delta Lake. Strong experience with the Medallion architecture (Bronze/Silver/Gold) data layer design pattern. Proficiency in Python, SQL, and PySpark. Experience with cloud platforms: AWS (preferred), Azure, or GCP. Hands-on experience with Delta Live Tables, Unity Catalog, and Lakehouse Federation is a plus. Knowledge of orchestration tools like Apache Airflow, Databricks Workflows, or Azure Data Factory. Familiarity with CI/CD tools (Git, Azure DevOps, Jenkins, etc.). Solid understanding of data security, RBAC/ABAC, and data governance practices. Strong analytical and problem-solving skills with attention to detail. Excellent verbal and written communication skills.
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.
Job Description: We are looking for an experienced Databricks Architect with deep expertise in Lakehouse and Medallion Architecture (Bronze, Silver, Gold layers) to lead the design and implementation of scalable, high-performance data solutions on the Databricks platform. The ideal candidate will have a strong background in big data architecture, data governance, and cloud-native data engineering using technologies like Delta Lake, Spark, and Apache Airflow.
Key Responsibilities : Design, architect, and implement end-to-end data pipelines using Databricks Lakehouse and Medallion architecture (Bronze, Silver, Gold layers). Lead development teams in building scalable and reusable frameworks for ingestion, transformation, and analytics. Define data modeling strategies and partitioning schemes to optimize performance and cost. Ensure data governance, quality, and security standards are incorporated throughout the architecture. Work closely with business stakeholders, data scientists, and analysts to align technical architecture with business objectives. Set up and manage CI/CD pipelines, orchestrators (e.g., Airflow, Databricks Workflows), and Delta Live Tables. Optimize Spark workloads and recommend performance tuning best practices for large-scale data processing. Act as a subject matter expert for Databricks, Delta Lake, Unity Catalog, and cloud-native data platforms (e.g., AWS, Azure, GCP). Conduct design reviews, code reviews, and technical workshops to mentor engineering teams. Required Skills and Qualifications:
Proven expertise with Databricks, Apache Spark, and Delta Lake. Strong experience with the Medallion architecture (Bronze/Silver/Gold) data layer design pattern. Proficiency in Python, SQL, and PySpark. Experience with cloud platforms: AWS (preferred), Azure, or GCP. Hands-on experience with Delta Live Tables, Unity Catalog, and Lakehouse Federation is a plus. Knowledge of orchestration tools like Apache Airflow, Databricks Workflows, or Azure Data Factory. Familiarity with CI/CD tools (Git, Azure DevOps, Jenkins, etc.). Solid understanding of data security, RBAC/ABAC, and data governance practices. Strong analytical and problem-solving skills with attention to detail. Excellent verbal and written communication skills.
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.