Galaxy i technologies Inc
Direct message the job poster from Galaxy i technologies Inc.
Employment Type:
Contract W2
Visa type:
Only US citizen and GC holders
Experience Level:
8+ years
About the Role We are seeking a highly skilled Data Engineer experienced in Scala and Apache Spark to design, build, and maintain scalable data pipelines and processing systems. You will collaborate with data scientists, analysts, and other engineers to ensure efficient data flow across our platforms and support business intelligence, analytics, and machine learning initiatives.
Key Responsibilities Design, develop, and optimize data pipelines using Apache Spark (batch and streaming).
Write efficient, scalable, and maintainable Scala code for ETL/ELT processes.
Integrate data from multiple structured and unstructured sources.
Work with data lake and data warehouse technologies (e.g., Delta Lake, Snowflake, Redshift, BigQuery).
Collaborate with cross-functional teams to ensure data quality, consistency, and governance.
Implement data validation, monitoring, and alerting frameworks.
Contribute to architecture design for large-scale data systems and distributed processing.
Optimize data processing performance and troubleshoot pipeline issues.
Follow best practices for code versioning, CI/CD, and DevOps in a data engineering context.
Required Skills & Qualifications Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
6+ years of experience in data engineering or backend software development.
Strong hands‑on experience with Apache Spark (DataFrame API, Spark SQL, Structured Streaming).
Proficiency in Scala (functional programming, performance optimization, testing).
Solid understanding of distributed systems, parallel processing, and data partitioning concepts.
Experience with data lakes, data warehouses, and ETL workflows.
Familiarity with cloud platforms (AWS, Azure, or GCP) and orchestration tools (Airflow, Databricks Jobs, etc.).
Working knowledge of SQL, NoSQL, and schema design.
Experience with CI/CD pipelines, Git, and containerization (Docker/Kubernetes).
Preferred / Nice-to-Have Experience with Databricks, Delta Lake, or Glue.
Knowledge of Python or Java in data engineering contexts.
Exposure to Kafka, Kinesis, or other streaming frameworks.
Understanding of data governance, security, and privacy regulations (GDPR, CCPA).
Familiarity with machine learning pipelines or feature engineering.
Soft Skills Strong problem‑solving and analytical skills.
Excellent communication and teamwork abilities.
Proactive, detail‑oriented, and able to work independently.
Passion for data‑driven decision‑making and scalable design.
#J-18808-Ljbffr
Employment Type:
Contract W2
Visa type:
Only US citizen and GC holders
Experience Level:
8+ years
About the Role We are seeking a highly skilled Data Engineer experienced in Scala and Apache Spark to design, build, and maintain scalable data pipelines and processing systems. You will collaborate with data scientists, analysts, and other engineers to ensure efficient data flow across our platforms and support business intelligence, analytics, and machine learning initiatives.
Key Responsibilities Design, develop, and optimize data pipelines using Apache Spark (batch and streaming).
Write efficient, scalable, and maintainable Scala code for ETL/ELT processes.
Integrate data from multiple structured and unstructured sources.
Work with data lake and data warehouse technologies (e.g., Delta Lake, Snowflake, Redshift, BigQuery).
Collaborate with cross-functional teams to ensure data quality, consistency, and governance.
Implement data validation, monitoring, and alerting frameworks.
Contribute to architecture design for large-scale data systems and distributed processing.
Optimize data processing performance and troubleshoot pipeline issues.
Follow best practices for code versioning, CI/CD, and DevOps in a data engineering context.
Required Skills & Qualifications Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
6+ years of experience in data engineering or backend software development.
Strong hands‑on experience with Apache Spark (DataFrame API, Spark SQL, Structured Streaming).
Proficiency in Scala (functional programming, performance optimization, testing).
Solid understanding of distributed systems, parallel processing, and data partitioning concepts.
Experience with data lakes, data warehouses, and ETL workflows.
Familiarity with cloud platforms (AWS, Azure, or GCP) and orchestration tools (Airflow, Databricks Jobs, etc.).
Working knowledge of SQL, NoSQL, and schema design.
Experience with CI/CD pipelines, Git, and containerization (Docker/Kubernetes).
Preferred / Nice-to-Have Experience with Databricks, Delta Lake, or Glue.
Knowledge of Python or Java in data engineering contexts.
Exposure to Kafka, Kinesis, or other streaming frameworks.
Understanding of data governance, security, and privacy regulations (GDPR, CCPA).
Familiarity with machine learning pipelines or feature engineering.
Soft Skills Strong problem‑solving and analytical skills.
Excellent communication and teamwork abilities.
Proactive, detail‑oriented, and able to work independently.
Passion for data‑driven decision‑making and scalable design.
#J-18808-Ljbffr