Compunnel, Inc.
We are seeking a highly experienced Cloud DevOps Engineer with a minimum of 10 years of hands-on expertise in cloud infrastructure, automation, and data engineering.
The ideal candidate will architect and manage scalable data solutions within an Azure Data Lakehouse environment, leveraging Databricks and other Azure services.
Key Responsibilities
Architect, configure, and optimize Databricks Pipelines for large-scale data processing in Azure Data Lakehouse. Set up and manage Azure infrastructure components including Databricks Workspaces, Azure Containers (AKS/ACI), Storage Accounts, and Networking. Design and implement monitoring and observability frameworks using Azure Monitor, Log Analytics, Prometheus, and Grafana. Collaborate with platform and data engineering teams to support microservices-based architecture for scalable data solutions. Drive automation and CI/CD practices using Terraform, ARM templates, GitHub Actions, and Azure DevOps. Required Qualifications
Minimum 10 years of hands-on experience in cloud and DevOps technologies. Strong expertise in Azure Databricks, Delta Lake, and Apache Spark. Deep understanding of Azure services including Resource Manager, AKS, ACR, Key Vault, and Networking. Proven experience in microservices architecture and container orchestration. Proficiency in infrastructure-as-code, scripting (Python, Bash), and DevOps tooling. Familiarity with cloud data governance, security, and cost optimization. Preferred Qualifications
Experience with event-driven architectures such as Kafka or Event Grid. Knowledge of data mesh principles and distributed data ownership.
#J-18808-Ljbffr
Architect, configure, and optimize Databricks Pipelines for large-scale data processing in Azure Data Lakehouse. Set up and manage Azure infrastructure components including Databricks Workspaces, Azure Containers (AKS/ACI), Storage Accounts, and Networking. Design and implement monitoring and observability frameworks using Azure Monitor, Log Analytics, Prometheus, and Grafana. Collaborate with platform and data engineering teams to support microservices-based architecture for scalable data solutions. Drive automation and CI/CD practices using Terraform, ARM templates, GitHub Actions, and Azure DevOps. Required Qualifications
Minimum 10 years of hands-on experience in cloud and DevOps technologies. Strong expertise in Azure Databricks, Delta Lake, and Apache Spark. Deep understanding of Azure services including Resource Manager, AKS, ACR, Key Vault, and Networking. Proven experience in microservices architecture and container orchestration. Proficiency in infrastructure-as-code, scripting (Python, Bash), and DevOps tooling. Familiarity with cloud data governance, security, and cost optimization. Preferred Qualifications
Experience with event-driven architectures such as Kafka or Event Grid. Knowledge of data mesh principles and distributed data ownership.
#J-18808-Ljbffr