Get new jobs for this search by email

Sr. Big Data Engineer Weehawken, NJ Strong Banking Domain Experience Is Required Only Local candidates to NJ/NY Position type: W2 contract. As a Big Data Engineer, candidate will be instrumental in developing and optimizing big data technologies such as Apache Spark and Azure Databricks. Candidate will create robust data pipelines, ensure high data quality, and collaborate closely with cross-functional teams to implement scalable cloud-native solutions on major cloud providers, predominantly Microsoft Azure. Sr. Big Data Engineer Weehawken, NJ Strong Banking Domain Experience Is Required Only Local candidates to NJ/NY Position type: W2 contract. Our Challenge As a Big Data Engineer, candidate will be instrumental in developing and optimizing big data technologies such as Apache Spark and Azure Databricks. Candidate will create robust data pipelines, ensure high data quality, and collaborate closely with cross-functional teams to implement scalable cloud-native solutions on major cloud providers, predominantly Microsoft Azure. The Role Responsibilities Develop, optimize, and maintain big data pipelines using Apache Spark, Azure Databricks, and related tools. Write and deploy complex production systems in Scala, Java, or Python, ensuring high performance and reliability. Leverage Terraform for automating cloud infrastructure deployment on Azure or other major cloud providers. Design and implement CI/CD pipelines for data workflows using modern tools like GitLab, promoting automation and continuous improvement. Apply best practices for testing, instrumentation, observability, and alerting to maintain data pipeline health and performance. Contribute to system architecture and low-level design, ensuring modularity, scalability, and security. Understand and implement data models, data structures, and algorithms for efficient data processing. Use containerization technologies such as Docker and Kubernetes for development, build, and runtime environments. Work within Agile development teams, participating in planning, daily stand-ups, and iterative releases. Collaborate effectively in a global team, influencing key architectural decisions and sharing best practices. Requirements 10+ Years of experience is required. Proven experience developing big data solutions with Apache Spark and Azure Databricks. Hands-on experience with Terraform for infrastructure as code (IaC) on major cloud platforms, ideally Azure. Programming expertise in Scala, Java, or Python for complex production systems. Extensive Python experience, especially in data engineering contexts. Background in platform engineering roles on cloud platforms, with strong knowledge of Azure services. Practical knowledge of testing frameworks, instrumentation, observability, and alerting tools. Experience building and maintaining CI/CD pipelines with modern cloud-friendly systems like GitLab. Deep understanding of information modelling, data structures, and algorithms. Hands-on experience with containerization (Docker) and orchestration tools (Kubernetes). Strong understanding of technical architecture and low-level system design. Familiarity with Agile methodologies and best practices in software development. Preferred, But Not Required Experience working with financial or risk management data. Knowledge of data governance, security standards, and compliance in cloud environments. Familiarity with additional cloud providers and multi-cloud deployment strategies. Contributions to open-source data projects or communities Seniority level Seniority level Mid-Senior level Employment type Employment type Contract Job function Job function Engineering and Information Technology Referrals increase your chances of interviewing at Accord Technologies Inc by 2x Sign in to set job alerts for “Big Data Developer” roles. Brooklyn, NY $100,000.00-$150,000.00 3 weeks ago Brooklyn, NY $120,000.00-$170,000.00 3 weeks ago New York City Metropolitan Area $150,000.00-$200,000.00 3 days ago New York, NY $170,000.00-$240,000.00 4 months ago New York, NY $139,500.00-$271,566.94 1 week ago New York, NY $130,000.00-$200,000.00 1 month ago Purchase, NY $100,000.00-$167,500.00 5 days ago New York, NY $90,000.00-$130,000.00 4 months ago New York, NY $110,000.00-$120,000.00 1 month ago New York City Metropolitan Area 1 week ago New York, NY $114,000.00-$171,000.00 2 weeks ago New York, NY $80,000.00-$160,000.00 1 week ago New York, NY $120,000.00-$121,000.00 2 days ago New York, NY $130,000.00-$200,000.00 7 months ago New York, NY $70,000.00-$100,000.00 1 week ago Data Engineer III, Go-to-Market Operations New York, NY $120,000.00-$132,000.00 4 days ago New York, NY $80,000.00-$100,000.00 1 week ago New York, NY $139,500.00-$271,566.94 1 week ago Data Engineer with Python / SQL - Onsite (Fulltime) New York, NY $69,661.00-$125,000.00 2 weeks ago New York, NY $124,000.00-$178,000.00 2 weeks ago New York, NY $100,000.00-$120,000.00 1 month ago We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI. #J-18808-Ljbffr