Nightwing
Databricks Engineer
Nightwing - Sterling, Virginia, United States, 22170
Work at Nightwing
Overview
- View job
Overview
Nightwing provides technically advanced full-spectrum cyber, data operations, systems integration and intelligence mission support services to meet our customers' most demanding challenges. Our capabilities include cyber space operations, cyber defense and resiliency, vulnerability research, ubiquitous technical surveillance, data intelligence, lifecycle mission enablement, and software modernization. Nightwing brings disruptive technologies, agility, and competitive offerings to customers in the intelligence community, defense, civil, and commercial markets.
Nightwing is supporting a U.S. Government customer on a large mission critical development and sustainment program to design, integrate, build, deliver, and operate a network operations environment including introducing new cyber capabilities to address emerging threats. Nightwing is seeking a Databricks Engineer to support the migration of customer applications, services and platforms to a Medallion Model. This opportunity represents the cornerstone of the future of the organization.
Responsibilities:
Supporting teams to migrate services, applications and platforms from legacy back-end systems to Databricks. Identifying the optimal path for migration, building the plan for migration and finally, execution of the plan. Data Pipeline migration of legacy pipelines from NiFi to Databricks, complete with validation. Implementing the medallion model for each of the data assets being migrated to Databricks. Develop an SOP for integration of data assets into the Databricks platform focused on efficiency, instrumentation and performance. Optimize development, testing, monitoring and security for data assets being added to the Databricks platform. Develop and implement a strategy for optimizing migration and integration of data assets to the Databricks platform. Develop code using various programming and scripting languages to automate/optimize data ingestion, pipeline orchestration and improve data management processes. Ingest transparency, leveraging technologies such as AWS CloudWatch to identify places for measuring and gathering performance information on automated data pipelines
Ensure Data Engineering Team Standard Operating Procedures are appropriately captured and communicated across the team. Ensure technical correctness, timeliness and quality of delivery for the team. Demonstrate excellent oral and written communication skills to all levels of management and the customer.
Required Skills:
Must be a US Citizen Active Secret (S) clearance. Must be able to obtain a TS/SCI clearance Must be able to obtain DHS Suitability 8 + years of directly relevant software development experience required. Minimum of 5 years of experience performing data engineering work in a cloud environment. Experience with relational, noSQL and/or file-based storage (e.g. Databricks, Elastic, Postgres, S3, Athena, etc.) Experience working in a CICD Pipeline factory environment Working knowledge of Databricks, Cloud Relational Database Services, NiFi, AWS Redshift and Elasticsearch Desired Skills:
Experience with:
Databricks workflows Databricks Unity Catalog Databricks Autoloader Databricks Delta Live Tables/Delta Lake Databricks Workspace/Notebooks MLflow Apache Spark
Experience with collaboration tools including MS Teams, MS Outlook, MS SharePoint, and Confluence Amazon Web Services (AWS) Professional certification or equivalent. Excellent problem-solving and communication skills. Familiarity with CISA: Securing the Software Supply Chain Familiarity with CISA: Cybersecurity Best Practices Familiarity with CISA: Open-Source Software Security Familiarity with NIST SP 800-218, Secure Software Development Framework V1.1: Recommendations for Mitigating the Risk of Software Vulnerabilities Required Education:
Bachelor's degree in Software Engineering, Computer Science or a related discipline is required. [Ten (10) years of experience (for a total of six (18) or more years) may be substituted for a degree.]
Syndeo # - 3466
At Nightwing, we value collaboration and teamwork. You'll have the opportunity to work alongside talented individuals who are passionate about what they do. Together, we'll leverage our collective expertise to drive innovation, solve complex problems, and deliver exceptional results for our clients.
Thank you for considering joining us as we embark on this new journey and shape the future of cybersecurity and intelligence together as part of the Nightwing team.
Nightwing is An Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability or veteran status, age or any other federally protected class.
Nightwing is supporting a U.S. Government customer on a large mission critical development and sustainment program to design, integrate, build, deliver, and operate a network operations environment including introducing new cyber capabilities to address emerging threats. Nightwing is seeking a Databricks Engineer to support the migration of customer applications, services and platforms to a Medallion Model. This opportunity represents the cornerstone of the future of the organization.
Responsibilities:
Supporting teams to migrate services, applications and platforms from legacy back-end systems to Databricks. Identifying the optimal path for migration, building the plan for migration and finally, execution of the plan. Data Pipeline migration of legacy pipelines from NiFi to Databricks, complete with validation. Implementing the medallion model for each of the data assets being migrated to Databricks. Develop an SOP for integration of data assets into the Databricks platform focused on efficiency, instrumentation and performance. Optimize development, testing, monitoring and security for data assets being added to the Databricks platform. Develop and implement a strategy for optimizing migration and integration of data assets to the Databricks platform. Develop code using various programming and scripting languages to automate/optimize data ingestion, pipeline orchestration and improve data management processes. Ingest transparency, leveraging technologies such as AWS CloudWatch to identify places for measuring and gathering performance information on automated data pipelines
Ensure Data Engineering Team Standard Operating Procedures are appropriately captured and communicated across the team. Ensure technical correctness, timeliness and quality of delivery for the team. Demonstrate excellent oral and written communication skills to all levels of management and the customer.
Required Skills:
Must be a US Citizen Active Secret (S) clearance. Must be able to obtain a TS/SCI clearance Must be able to obtain DHS Suitability 8 + years of directly relevant software development experience required. Minimum of 5 years of experience performing data engineering work in a cloud environment. Experience with relational, noSQL and/or file-based storage (e.g. Databricks, Elastic, Postgres, S3, Athena, etc.) Experience working in a CICD Pipeline factory environment Working knowledge of Databricks, Cloud Relational Database Services, NiFi, AWS Redshift and Elasticsearch Desired Skills:
Experience with:
Databricks workflows Databricks Unity Catalog Databricks Autoloader Databricks Delta Live Tables/Delta Lake Databricks Workspace/Notebooks MLflow Apache Spark
Experience with collaboration tools including MS Teams, MS Outlook, MS SharePoint, and Confluence Amazon Web Services (AWS) Professional certification or equivalent. Excellent problem-solving and communication skills. Familiarity with CISA: Securing the Software Supply Chain Familiarity with CISA: Cybersecurity Best Practices Familiarity with CISA: Open-Source Software Security Familiarity with NIST SP 800-218, Secure Software Development Framework V1.1: Recommendations for Mitigating the Risk of Software Vulnerabilities Required Education:
Bachelor's degree in Software Engineering, Computer Science or a related discipline is required. [Ten (10) years of experience (for a total of six (18) or more years) may be substituted for a degree.]
Syndeo # - 3466
At Nightwing, we value collaboration and teamwork. You'll have the opportunity to work alongside talented individuals who are passionate about what they do. Together, we'll leverage our collective expertise to drive innovation, solve complex problems, and deliver exceptional results for our clients.
Thank you for considering joining us as we embark on this new journey and shape the future of cybersecurity and intelligence together as part of the Nightwing team.
Nightwing is An Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability or veteran status, age or any other federally protected class.