Qode
Senior Big Data Engineer
Qode - Charlotte, North Carolina, United States, 28245
Work at Qode
Overview
- View job
Overview
Role Description : Senior Big Data Engineer – Scala/Spark
Location:
[Charlotte NC/NY/NJ/or Onsite/Hybrid] Type:
[Full-time] Experience Level:
Senior level only Role Overview We are looking for a highly skilled
Senior Big Data Engineer
with strong expertise in
Scala
and
Apache Spark
to support our data engineering team. The ideal candidate should be able to read and refactor existing Scala Spark code, develop enhancements, create robust unit tests, and help migrate code into a modern cloud-native environment. Key Responsibilities: Analyze and refactor existing Scala Spark code to support migration to new environments. Write and enhance Scala Spark code with a focus on performance and maintainability. Develop comprehensive unit tests to ensure code quality and stability. Work with data stored in Amazon S3 (CSV and Parquet formats). Integrate and manipulate data from MongoDB. Independently set up environments to validate and test code changes. Execute both manual and automated testing procedures. Collaborate within an Agile team and contribute to sprint planning, story grooming, and reviews. Maintain clear and proactive communication with team members, project managers, and stakeholders. Technical Skills & Qualifications: Proficiency in Scala and Apache Spark for large-scale data processing. Working knowledge of Python for scripting and data transformation. Hands-on experience with Amazon Web Services (AWS), including: S3 AWS Glue Step Functions Experience working with MongoDB and structured/unstructured data. Familiarity with SQL and BDD frameworks. Experience with Terraform for infrastructure-as-code is a plus. Strong understanding of software development lifecycle (SDLC), from development to production deployment. Experience using tools like Git, Azure DevOps/TFS, and JIRA. Comfortable working in Agile/Scrum environments. Core Technologies: Scala Spark AWS Glue AWS Step Functions Maven Terraform Soft Skills: Strong problem-solving abilities and attention to detail. Ability to work independently and as part of a collaborative Agile team. Excellent written and verbal communication skills.
[Charlotte NC/NY/NJ/or Onsite/Hybrid] Type:
[Full-time] Experience Level:
Senior level only Role Overview We are looking for a highly skilled
Senior Big Data Engineer
with strong expertise in
Scala
and
Apache Spark
to support our data engineering team. The ideal candidate should be able to read and refactor existing Scala Spark code, develop enhancements, create robust unit tests, and help migrate code into a modern cloud-native environment. Key Responsibilities: Analyze and refactor existing Scala Spark code to support migration to new environments. Write and enhance Scala Spark code with a focus on performance and maintainability. Develop comprehensive unit tests to ensure code quality and stability. Work with data stored in Amazon S3 (CSV and Parquet formats). Integrate and manipulate data from MongoDB. Independently set up environments to validate and test code changes. Execute both manual and automated testing procedures. Collaborate within an Agile team and contribute to sprint planning, story grooming, and reviews. Maintain clear and proactive communication with team members, project managers, and stakeholders. Technical Skills & Qualifications: Proficiency in Scala and Apache Spark for large-scale data processing. Working knowledge of Python for scripting and data transformation. Hands-on experience with Amazon Web Services (AWS), including: S3 AWS Glue Step Functions Experience working with MongoDB and structured/unstructured data. Familiarity with SQL and BDD frameworks. Experience with Terraform for infrastructure-as-code is a plus. Strong understanding of software development lifecycle (SDLC), from development to production deployment. Experience using tools like Git, Azure DevOps/TFS, and JIRA. Comfortable working in Agile/Scrum environments. Core Technologies: Scala Spark AWS Glue AWS Step Functions Maven Terraform Soft Skills: Strong problem-solving abilities and attention to detail. Ability to work independently and as part of a collaborative Agile team. Excellent written and verbal communication skills.