Phaxis
This range is provided by Phaxis. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.
Base pay range $125,000.00/yr - $135,000.00/yr
Key Responsibilities
Design, develop, and maintain end-to-end data pipelines and ETL/ELT workflows using Python, SQL, and modern orchestration tools (e.g., Airflow, dbt).
Architect and optimize data storage solutions, including data lakes and data warehouses, using Databricks, Delta Lake, and cloud-native services.
Build scalable data processing solutions leveraging Databricks notebooks, jobs, and clusters for both batch and streaming data workloads.
Develop and manage Databricks workflows using Spark (PySpark, SQL, or Scala) to transform, cleanse, and aggregate large datasets.
Implement data quality checks, schema validation, and monitoring to ensure data accuracy and reliability.
Optimize Databricks cluster configurations and job performance to minimize cost and maximize throughput.
Collaborate with DevOps teams to automate deployments, CI/CD pipelines, and infrastructure-as-code (IaC) for data systems.
Qualifications
Bachelor's or Master's degree in Computer Science, Data Engineering, or a related technical field.
Advanced proficiency in Python and SQL for data manipulation, transformation, and automation.
Deep experience with Databricks, including Spark optimization, Delta Lake management, job orchestration, and workspace administration.
Strong understanding of distributed data processing, partitioning strategies, and performance tuning in Databricks and Spark.
Hands‑on experience with cloud platforms (AWS, Azure, or GCP) and their data ecosystems (e.g., S3, ADLS, BigQuery, Snowflake).
Seniorilty level Director
Employment type Full-time
Job function Information Technology
#J-18808-Ljbffr
Base pay range $125,000.00/yr - $135,000.00/yr
Key Responsibilities
Design, develop, and maintain end-to-end data pipelines and ETL/ELT workflows using Python, SQL, and modern orchestration tools (e.g., Airflow, dbt).
Architect and optimize data storage solutions, including data lakes and data warehouses, using Databricks, Delta Lake, and cloud-native services.
Build scalable data processing solutions leveraging Databricks notebooks, jobs, and clusters for both batch and streaming data workloads.
Develop and manage Databricks workflows using Spark (PySpark, SQL, or Scala) to transform, cleanse, and aggregate large datasets.
Implement data quality checks, schema validation, and monitoring to ensure data accuracy and reliability.
Optimize Databricks cluster configurations and job performance to minimize cost and maximize throughput.
Collaborate with DevOps teams to automate deployments, CI/CD pipelines, and infrastructure-as-code (IaC) for data systems.
Qualifications
Bachelor's or Master's degree in Computer Science, Data Engineering, or a related technical field.
Advanced proficiency in Python and SQL for data manipulation, transformation, and automation.
Deep experience with Databricks, including Spark optimization, Delta Lake management, job orchestration, and workspace administration.
Strong understanding of distributed data processing, partitioning strategies, and performance tuning in Databricks and Spark.
Hands‑on experience with cloud platforms (AWS, Azure, or GCP) and their data ecosystems (e.g., S3, ADLS, BigQuery, Snowflake).
Seniorilty level Director
Employment type Full-time
Job function Information Technology
#J-18808-Ljbffr