Purple Drive
Key Responsibilities:
Design, build, and optimize scalable data pipelines and frameworks on
AWS
leveraging
EMR, EKS, S3, and related services . Develop, deploy, and manage
Databricks (DBX) workflows
for big data processing, analytics, and machine learning. Implement infrastructure as code (IaC) solutions using
Terraform
for automated provisioning and environment consistency. Write efficient, reusable, and optimized code in
Scala, Python, and Java
for data transformation, streaming, and integration. Collaborate with architects, data scientists, and business stakeholders to translate requirements into technical solutions. Ensure
performance tuning, scalability, and cost optimization
across cloud-based data platforms. Enforce best practices in
security, CI/CD, testing, and DevOps integration
for cloud-native applications. Required Skills & Experience:
7-10 years of hands-on experience in
data engineering and cloud frameworks . Proven expertise with
AWS big data services
(EMR, EKS, S3, Lambda, Glue). Strong development skills in
Scala, Python, and Java . Deep knowledge of
Databricks (DBX)
for batch and streaming workloads. Solid experience with
Terraform
for infrastructure automation. Familiarity with CI/CD, containerization (Docker, Kubernetes), and Agile methodologies. Nice-to-Have Skills:
Exposure to
machine learning workflows
on Databricks. Knowledge of
Airflow or other orchestration tools . Experience with
data lakehouse architectures .
Design, build, and optimize scalable data pipelines and frameworks on
AWS
leveraging
EMR, EKS, S3, and related services . Develop, deploy, and manage
Databricks (DBX) workflows
for big data processing, analytics, and machine learning. Implement infrastructure as code (IaC) solutions using
Terraform
for automated provisioning and environment consistency. Write efficient, reusable, and optimized code in
Scala, Python, and Java
for data transformation, streaming, and integration. Collaborate with architects, data scientists, and business stakeholders to translate requirements into technical solutions. Ensure
performance tuning, scalability, and cost optimization
across cloud-based data platforms. Enforce best practices in
security, CI/CD, testing, and DevOps integration
for cloud-native applications. Required Skills & Experience:
7-10 years of hands-on experience in
data engineering and cloud frameworks . Proven expertise with
AWS big data services
(EMR, EKS, S3, Lambda, Glue). Strong development skills in
Scala, Python, and Java . Deep knowledge of
Databricks (DBX)
for batch and streaming workloads. Solid experience with
Terraform
for infrastructure automation. Familiarity with CI/CD, containerization (Docker, Kubernetes), and Agile methodologies. Nice-to-Have Skills:
Exposure to
machine learning workflows
on Databricks. Knowledge of
Airflow or other orchestration tools . Experience with
data lakehouse architectures .