Ampstek
Overview
We are seeking an experienced Senior DataOps Engineer to join our team. This candidate will have a strong background in DevOps, DataOps, or Cloud Engineering practices, with extensive experience in automating the CICD pipelines and modern data stack technologies. Key Responsibilities
Develop and maintain robust, scalable data pipelines and infrastructure automation workflows using GitHub, AWS, and Databricks. Implement and manage CI/CD pipelines using GitHub Actions and GitLab CI/CD for automated infrastructure deployment, testing, and validation. Deploy and manage Databricks LLM Runtime or custom Hugging Face models within Databricks notebooks and model serving endpoints. Manage and optimize Cloud Infrastructure costs, usage, and performance through tagging policies, right-sizing EC2 instances, storage tiering strategies, and auto-scaling. Set up infrastructure observability and performance dashboards using AWS CloudWatch for real-time insights into cloud resources and data pipelines. Develop and manage Terraform or CloudFormation modules to automate infrastructure provisioning across AWS accounts and environments. Implement and enforce cloud security policies, IAM roles, encryption mechanisms (KMS), and compliance configurations. Administer Databricks Workspaces, clusters, access controls, and integrations with Cloud Storage and identity providers. Enforce DevSecOps practices for infrastructure-as-code, ensuring all changes are peer-reviewed, tested, and compliant with internal security policies. Coordinate cloud software releases, patching schedules, and vulnerability remediations using Systems Manager Patch Manage. Automate AWS housekeeping and operational tasks such as cleanup of unused EBS volumes and snapshots, old AMIs; rotation of secrets and credentials; log retention enforcement. Perform incident response, disaster recovery planning, and post-mortem analysis for operational outages. Collaborate with cross-functional teams including Data Scientists, Data Engineers, and other stakeholders to gather and implement infrastructure and data requirements. Required Skills and Experience
8+ years of experience in DataOps / CloudOps / DevOps roles, with strong focus on infrastructure automation, data pipeline operations, observability, and cloud administration. Strong proficiency in at least one scripting language (e.g., Python, Bash) and one infrastructure-as-code tool (e.g., Terraform, CloudFormation) for building automation scripts for AWS resource cleanup, tagging enforcement, monitoring and backups. Hands-on experience integrating and operationalizing LLMs in production pipelines, including prompt management, caching, token-tracking, and post-processing. Deep hands-on experience with AWS Services including EC2, S3, RDS, CloudWatch, IAM, Lambda, VPC; Data Services: Athena, Glue, MSK, Redshift; Security: KMS, IAM, Config, CloudTrail, Secrets Manager; Operational: Auto Scaling, Systems Manager, CloudFormation/Terraform. Working knowledge of Databricks, including cluster and workspace management, job orchestration, and integration with AWS storage and identity (IAM passthrough). Experience deploying and managing CI/CD workflows using GitHub Actions, GitLab CI, or AWS CodePipeline. Strong understanding of cloud networking (VPC Peering, Transit Gateway, security groups, private link). Familiarity with container orchestration platforms (Kubernetes, ECS) for deploying platform tools and services. Strong understanding of data modeling, data warehousing concepts, and AI/ML lifecycle management. Knowledge of cost optimization strategies across compute, storage, and network layers. Experience with data governance, logging, and compliance practices in cloud environments (SOC2, HIPAA, GDPR). Bonus: Exposure to LangChain, Prompt Engineering frameworks, Retrieval Augmented Generation (RAG), and vector database integration (AWS OpenSearch, Pinecone, Milvus, etc.). Preferred Qualifications
AWS Certified Solutions Architect, DevOps Engineer, or SysOps Administrator certifications. Hands-on experience with multi-cloud environments, particularly Azure or GCP, in addition to AWS. Experience with infrastructure cost management tools like AWS Cost Explorer, or FinOps dashboards. Ability to write clean, production-grade Python code for automation scripts, operational tooling, and custom CloudOps utilities. Prior experience in supporting high-availability production environments with disaster recovery and failover architectures. Understanding of Zero Trust architecture and security best practices in cloud-native environments. Experience with automated cloud resources cleanup, tagging enforcement, and compliance-as-code using tools like Terraform Sentinel. Familiarity with Databricks Unity Catalog, access control frameworks, and workspace governance. Strong communication skills and experience working in agile cross-functional teams, ideally with Data Product or Platform Engineering teams. About Ampstek
Ampstek is a global IT solutions partner serving clients across North America, Europe, APAC, LATAM, and MEA. We specialize in delivering talent and technology solutions for enterprise-level digital transformation, trading systems, data services, and regulatory compliance. Job details
Employment type: Contract Seniority level: Mid-Senior level Job function: Information Technology Industries: IT Services and IT Consulting Note
Referrals increase your chances of interviewing at Ampstek. We’re not providing additional job postings beyond this listing here.
#J-18808-Ljbffr
We are seeking an experienced Senior DataOps Engineer to join our team. This candidate will have a strong background in DevOps, DataOps, or Cloud Engineering practices, with extensive experience in automating the CICD pipelines and modern data stack technologies. Key Responsibilities
Develop and maintain robust, scalable data pipelines and infrastructure automation workflows using GitHub, AWS, and Databricks. Implement and manage CI/CD pipelines using GitHub Actions and GitLab CI/CD for automated infrastructure deployment, testing, and validation. Deploy and manage Databricks LLM Runtime or custom Hugging Face models within Databricks notebooks and model serving endpoints. Manage and optimize Cloud Infrastructure costs, usage, and performance through tagging policies, right-sizing EC2 instances, storage tiering strategies, and auto-scaling. Set up infrastructure observability and performance dashboards using AWS CloudWatch for real-time insights into cloud resources and data pipelines. Develop and manage Terraform or CloudFormation modules to automate infrastructure provisioning across AWS accounts and environments. Implement and enforce cloud security policies, IAM roles, encryption mechanisms (KMS), and compliance configurations. Administer Databricks Workspaces, clusters, access controls, and integrations with Cloud Storage and identity providers. Enforce DevSecOps practices for infrastructure-as-code, ensuring all changes are peer-reviewed, tested, and compliant with internal security policies. Coordinate cloud software releases, patching schedules, and vulnerability remediations using Systems Manager Patch Manage. Automate AWS housekeeping and operational tasks such as cleanup of unused EBS volumes and snapshots, old AMIs; rotation of secrets and credentials; log retention enforcement. Perform incident response, disaster recovery planning, and post-mortem analysis for operational outages. Collaborate with cross-functional teams including Data Scientists, Data Engineers, and other stakeholders to gather and implement infrastructure and data requirements. Required Skills and Experience
8+ years of experience in DataOps / CloudOps / DevOps roles, with strong focus on infrastructure automation, data pipeline operations, observability, and cloud administration. Strong proficiency in at least one scripting language (e.g., Python, Bash) and one infrastructure-as-code tool (e.g., Terraform, CloudFormation) for building automation scripts for AWS resource cleanup, tagging enforcement, monitoring and backups. Hands-on experience integrating and operationalizing LLMs in production pipelines, including prompt management, caching, token-tracking, and post-processing. Deep hands-on experience with AWS Services including EC2, S3, RDS, CloudWatch, IAM, Lambda, VPC; Data Services: Athena, Glue, MSK, Redshift; Security: KMS, IAM, Config, CloudTrail, Secrets Manager; Operational: Auto Scaling, Systems Manager, CloudFormation/Terraform. Working knowledge of Databricks, including cluster and workspace management, job orchestration, and integration with AWS storage and identity (IAM passthrough). Experience deploying and managing CI/CD workflows using GitHub Actions, GitLab CI, or AWS CodePipeline. Strong understanding of cloud networking (VPC Peering, Transit Gateway, security groups, private link). Familiarity with container orchestration platforms (Kubernetes, ECS) for deploying platform tools and services. Strong understanding of data modeling, data warehousing concepts, and AI/ML lifecycle management. Knowledge of cost optimization strategies across compute, storage, and network layers. Experience with data governance, logging, and compliance practices in cloud environments (SOC2, HIPAA, GDPR). Bonus: Exposure to LangChain, Prompt Engineering frameworks, Retrieval Augmented Generation (RAG), and vector database integration (AWS OpenSearch, Pinecone, Milvus, etc.). Preferred Qualifications
AWS Certified Solutions Architect, DevOps Engineer, or SysOps Administrator certifications. Hands-on experience with multi-cloud environments, particularly Azure or GCP, in addition to AWS. Experience with infrastructure cost management tools like AWS Cost Explorer, or FinOps dashboards. Ability to write clean, production-grade Python code for automation scripts, operational tooling, and custom CloudOps utilities. Prior experience in supporting high-availability production environments with disaster recovery and failover architectures. Understanding of Zero Trust architecture and security best practices in cloud-native environments. Experience with automated cloud resources cleanup, tagging enforcement, and compliance-as-code using tools like Terraform Sentinel. Familiarity with Databricks Unity Catalog, access control frameworks, and workspace governance. Strong communication skills and experience working in agile cross-functional teams, ideally with Data Product or Platform Engineering teams. About Ampstek
Ampstek is a global IT solutions partner serving clients across North America, Europe, APAC, LATAM, and MEA. We specialize in delivering talent and technology solutions for enterprise-level digital transformation, trading systems, data services, and regulatory compliance. Job details
Employment type: Contract Seniority level: Mid-Senior level Job function: Information Technology Industries: IT Services and IT Consulting Note
Referrals increase your chances of interviewing at Ampstek. We’re not providing additional job postings beyond this listing here.
#J-18808-Ljbffr