ConsultNet Technology Services and Solutions
Big Data Engineer
ConsultNet Technology Services and Solutions, Rockville, Maryland, us, 20849
Title : Big Data Engineer
Location
: Rockville, Maryland (hybrid) Target Start Date
: May 15, 2025 Type : contract Pay
Rate : DOE
We are looking for an experienced Big Data Engineer to lead the modernization and development of scalable data pipelines that support market regulation and surveillance applications. This role will focus on building efficient, cloud-native ETL systems, enabling data science experimentation, and ensuring high performance and governance in data workflows. Key Responsibilities:
Design and modernize modular, scalable data pipelines from legacy systems.
Build high-performance ETL workflows using Apache Spark and AWS-based tools.
Enable no-code/low-code experimentation through configurable pipeline frameworks.
Optimize compute performance, reduce storage costs, and maintain data integrity.
Ensure compliance with governance, lineage, and security standards.
Integrate with enterprise data lakes and cloud storage platforms.
Implement CI/CD pipelines and automated testing (unit, integration, end-to-end).
Collaborate in Agile teams, participating in sprints and peer reviews.
Required Qualifications:
Bachelor's degree in Computer Science, Software Engineering, or a related field.
7+ years of experience in large-scale data engineering or infrastructure roles.
Proficient in Python or Scala for pipeline development.
Deep expertise in Apache Spark, Hive, and distributed data processing.
Strong SQL and NoSQL skills (e.g., PostgreSQL, MongoDB, DynamoDB).
Hands-on experience with AWS services for compute, storage, and orchestration.
Familiarity with CI/CD practices and tools like Jenkins.
Knowledge of orchestration frameworks (e.g., Apache Airflow).
Experience with feature stores and ML pipeline integration.
Preferred:
Experience with financial data, regulatory reporting, or trade analytics.
Background in machine learning feature engineering and model deployment
READ, UPDATE and DELETE
- When putting your job descriptions into Job Diva please use Calibri for the Font and size 11. We also do not underline anything, bold anything, or have any variation in color. We also bullet point the required and desired skills no dashes, arrows, etc.
SPELL-CHECK EVERYTHING BEFORE YOU CLICK SAVE!
Welcome to ConsultNet and the family of companies, Tekne, SaltClick, TechBridge, and OmniMedia. As a premier national provider of technology talent and solutions, our expertise spans across project services, contract-to-hire, direct placement, and managed services both onshore and nearshore.
Celebrating more than 25 years of partnership with a diverse client base, we've crafted rewarding opportunities for our consultants, fostering high-performing teams that deliver impactful results. Over the last few years thousands of consultants have found their calling with us in roles that have made a meaningful impact on their lives, enhanced their career, challenged them, and propelled them towards achieving their personal and professional goals. At the ConsultNet family of companies, we believe effective communication is crucial in aligning the right job with your unique skills and professional aspirations. To us, it's all about the personal approach we take and the values we uphold. Our comprehensive service offerings cover a wide range of technology positions across key markets nationwide. Client more at www.consultnet.com . We champion equality and inclusivity, proudly supporting an Equal Opportunity Employer policy. We welcome applicants regardless of Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other status protected by law. #J-18808-Ljbffr
: Rockville, Maryland (hybrid) Target Start Date
: May 15, 2025 Type : contract Pay
Rate : DOE
We are looking for an experienced Big Data Engineer to lead the modernization and development of scalable data pipelines that support market regulation and surveillance applications. This role will focus on building efficient, cloud-native ETL systems, enabling data science experimentation, and ensuring high performance and governance in data workflows. Key Responsibilities:
Design and modernize modular, scalable data pipelines from legacy systems.
Build high-performance ETL workflows using Apache Spark and AWS-based tools.
Enable no-code/low-code experimentation through configurable pipeline frameworks.
Optimize compute performance, reduce storage costs, and maintain data integrity.
Ensure compliance with governance, lineage, and security standards.
Integrate with enterprise data lakes and cloud storage platforms.
Implement CI/CD pipelines and automated testing (unit, integration, end-to-end).
Collaborate in Agile teams, participating in sprints and peer reviews.
Required Qualifications:
Bachelor's degree in Computer Science, Software Engineering, or a related field.
7+ years of experience in large-scale data engineering or infrastructure roles.
Proficient in Python or Scala for pipeline development.
Deep expertise in Apache Spark, Hive, and distributed data processing.
Strong SQL and NoSQL skills (e.g., PostgreSQL, MongoDB, DynamoDB).
Hands-on experience with AWS services for compute, storage, and orchestration.
Familiarity with CI/CD practices and tools like Jenkins.
Knowledge of orchestration frameworks (e.g., Apache Airflow).
Experience with feature stores and ML pipeline integration.
Preferred:
Experience with financial data, regulatory reporting, or trade analytics.
Background in machine learning feature engineering and model deployment
READ, UPDATE and DELETE
- When putting your job descriptions into Job Diva please use Calibri for the Font and size 11. We also do not underline anything, bold anything, or have any variation in color. We also bullet point the required and desired skills no dashes, arrows, etc.
SPELL-CHECK EVERYTHING BEFORE YOU CLICK SAVE!
Welcome to ConsultNet and the family of companies, Tekne, SaltClick, TechBridge, and OmniMedia. As a premier national provider of technology talent and solutions, our expertise spans across project services, contract-to-hire, direct placement, and managed services both onshore and nearshore.
Celebrating more than 25 years of partnership with a diverse client base, we've crafted rewarding opportunities for our consultants, fostering high-performing teams that deliver impactful results. Over the last few years thousands of consultants have found their calling with us in roles that have made a meaningful impact on their lives, enhanced their career, challenged them, and propelled them towards achieving their personal and professional goals. At the ConsultNet family of companies, we believe effective communication is crucial in aligning the right job with your unique skills and professional aspirations. To us, it's all about the personal approach we take and the values we uphold. Our comprehensive service offerings cover a wide range of technology positions across key markets nationwide. Client more at www.consultnet.com . We champion equality and inclusivity, proudly supporting an Equal Opportunity Employer policy. We welcome applicants regardless of Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other status protected by law. #J-18808-Ljbffr