M1 Technology
Chantilly, Virginia, United States; Herndon, Virginia, United States
We are seeking a
Senior Data Engineer
to lead the development and optimization of data pipelines, data movement tools, and data preparation services to support advanced analytics and enterprise integration. This role requires deep expertise in handling both structured and unstructured data, designing scalable data architectures, and developing reliable ETL solutions. The ideal candidate has significant experience with both cloud-native and on-premise technologies, and thrives in collaborative, mission-driven environments. Clearance Requirement: TS/SCI with Polygraph Key Responsibilities:
Design, implement, and maintain scalable data pipelines using tools such as Spark, Apache Iceberg, Trino, OpenSearch, EMR, NiFi, and Kubernetes Perform data ingestion, extraction, formatting, and transformation for a wide range of data types and formats Clean, preprocess, and format data for exploration, analysis, and operational use Develop ETL tools and scripts and enhance existing pipelines using coding best practices Work with Data Scientists, analysts, and mission partners to align data pipelines with mission goals Collaborate with software engineers to configure and maintain data services and back-end components Ensure data quality, standardization, and provenance through collaboration with testing and data quality teams Produce comprehensive documentation including ETL mappings, data lineage, code usage guides, and access instructions Participate in enterprise working groups to define and promote data standards across systems Support both one-time and ongoing data extraction and transformation efforts across various repositories Required Qualifications:
10+ years of experience in data lifecycle engineering Deep expertise in ETL development, pipeline design, and data architecture Strong coding skills in Python, SQL, Spark, and other data engineering tools Experience with COTS and open-source tools including ElasticSearch and Apache NiFi Strong understanding of cloud and on-premise data storage and processing solutions Experience working in Agile development environments Proven track record of integrating diverse data sources into enterprise systems Preferred Qualifications:
Experience deploying solutions on cloud platforms (e.g., AWS EMR, S3, RDS, EKS) Familiarity with data governance, data lineage tracking, and security requirements Hands-on experience with container orchestration using Kubernetes Knowledge of data lake architectures and streaming data processing Excellent communication skills and experience collaborating with cross-functional teams Languages & Frameworks: Python, SQL, Spark DevOps & CI/CD: Git, Agile methodologies, automated testing frameworks Why Join M1?
Tackle large-scale data engineering challenges with national impact Work with cutting-edge cloud and open-source data tools Collaborate with a multidisciplinary team of engineers and analysts Flexible, mission-driven environment with opportunities for leadership M1 Technology is an equal opportunity employer and values diversity. We do not discriminate in hiring on the basis of race, color, religion, sex, national origin, age, disability, veteran status, or any other characteristic protected by federal, state, or local law.
#J-18808-Ljbffr
Senior Data Engineer
to lead the development and optimization of data pipelines, data movement tools, and data preparation services to support advanced analytics and enterprise integration. This role requires deep expertise in handling both structured and unstructured data, designing scalable data architectures, and developing reliable ETL solutions. The ideal candidate has significant experience with both cloud-native and on-premise technologies, and thrives in collaborative, mission-driven environments. Clearance Requirement: TS/SCI with Polygraph Key Responsibilities:
Design, implement, and maintain scalable data pipelines using tools such as Spark, Apache Iceberg, Trino, OpenSearch, EMR, NiFi, and Kubernetes Perform data ingestion, extraction, formatting, and transformation for a wide range of data types and formats Clean, preprocess, and format data for exploration, analysis, and operational use Develop ETL tools and scripts and enhance existing pipelines using coding best practices Work with Data Scientists, analysts, and mission partners to align data pipelines with mission goals Collaborate with software engineers to configure and maintain data services and back-end components Ensure data quality, standardization, and provenance through collaboration with testing and data quality teams Produce comprehensive documentation including ETL mappings, data lineage, code usage guides, and access instructions Participate in enterprise working groups to define and promote data standards across systems Support both one-time and ongoing data extraction and transformation efforts across various repositories Required Qualifications:
10+ years of experience in data lifecycle engineering Deep expertise in ETL development, pipeline design, and data architecture Strong coding skills in Python, SQL, Spark, and other data engineering tools Experience with COTS and open-source tools including ElasticSearch and Apache NiFi Strong understanding of cloud and on-premise data storage and processing solutions Experience working in Agile development environments Proven track record of integrating diverse data sources into enterprise systems Preferred Qualifications:
Experience deploying solutions on cloud platforms (e.g., AWS EMR, S3, RDS, EKS) Familiarity with data governance, data lineage tracking, and security requirements Hands-on experience with container orchestration using Kubernetes Knowledge of data lake architectures and streaming data processing Excellent communication skills and experience collaborating with cross-functional teams Languages & Frameworks: Python, SQL, Spark DevOps & CI/CD: Git, Agile methodologies, automated testing frameworks Why Join M1?
Tackle large-scale data engineering challenges with national impact Work with cutting-edge cloud and open-source data tools Collaborate with a multidisciplinary team of engineers and analysts Flexible, mission-driven environment with opportunities for leadership M1 Technology is an equal opportunity employer and values diversity. We do not discriminate in hiring on the basis of race, color, religion, sex, national origin, age, disability, veteran status, or any other characteristic protected by federal, state, or local law.
#J-18808-Ljbffr