Virginia Jobs
CO - Senior Data Engineer & ETL Architect
Virginia Jobs, Richmond, Virginia, United States, 23214
Hiring Range:
$110,000-$135,000
Full Time or Part Time:
FULL TIME Additional Detail JOB DESCRIPTION Serve as a senior-level data engineer and ETL developer responsible for modernizing and sustaining data integration systems across the enterprise. This position also functions as the lead technical data architect on the OIS (Office of Information Services) architecture team, guiding the design and implementation of future-forward data platforms, pipelines, and integrations. The role is critical in bridging support for legacy SSIS-based systems with development of cutting-edge data engineering solutions, ensuring continuity, scalability, and innovation. RESPONSIBILITIES
Design, develop, and maintain both legacy ETL processes (SSIS) and modern data pipelines and orchestration mechanisms.
Architect scalable transactional and data warehouse solutions using relational, columnar, and cloud-based data stores.
Partner with cross-functional teams to define data architecture standards, integration patterns, and governance strategies.
Lead the technical evolution of enterprise data infrastructure in support of analytics, reporting, and system interoperability with transactional / OLTP data sources.
Work collaboratively with cross-functional teams in OIS, Office of Process Improvement and Data Applications (PIDA), and Office of Business Transformation (BT) to support and develop high functioning data integrations and related service levels for both transactional and analytical data platforms.
Provide mentorship and technical leadership across the data / analytics and application development teams.
Manage metadata, data lineage, and version/source control across data pipelines.
Work closely with agency IT Security and Data Governance programs to ensure data ETL follows agency and Commonwealth data governance and security standards and policies.
Serve as a point of escalation and guidance for performance tuning, troubleshooting, issue resolution and optimizations of legacy and modern data ecosystems.
QUALIFICATIONS
Minimum 8 years of combined experience in data engineering, data integration, or database architecture.
Proven record of architecting, developing and deploying enterprise-grade data solutions that span operational/transactional and analytical domains.
Expertise in logical and physical database design, data modeling (ER dimensional), and metadata management.
Proficiency in RDBMS design and development (SQL Server and Oracle required), including T-SQL, PL/SQL, and performance tuning; experience with database replication.
High-level proficiency with Python as a scripting and ETL automation language.
Demonstrated ability in developing and supporting ETL/ELT solutions across real-time and batch processing paradigms.
Expertise with legacy integration tools (SSIS) and modern ETL/data orchestration tools (Apache Airflow, Dagster, Airbyte, Perfect, etc.).
Experience with large-scale migrations/conversions; experience with Oracle-to-SQL Server conversion tools is a plus.
Familiarity with modern data warehouse ecosystems (dbt, Clickhouse, Snowflake, Databricks, Fivetran, etc.) in cloud environments.
Strong understanding of data warehousing, data lakes and data mesh patterns.
Excellent communication skills for working across diverse technical and non-technical teams.
Deep experience with SDLC and agile data engineering methodologies.
Ability to balance legacy system support with forward-looking architectural vision.
Experience integrating with RESTful APIs using JSON and implementing message-based architectures (e.g., RabbitMQ, Apache Kafka, Azure Service Bus) is a plus.
Prior experience in a technical leadership or enterprise architecture role preferred.
ABOUT US The Virginia Department of Environmental Quality (DEQ) is the primary environmental permitting agency in the Commonwealth of Virginia. It is responsible for administering laws and regulations related to air quality, water quality, water supply, renewable energy, and land protection. Through the dedication and work of over 800 employees across six regional offices and the Central Office in Richmond, DEQ issues permits, conducts monitoring and inspections, and enforces the law.
$110,000-$135,000
Full Time or Part Time:
FULL TIME Additional Detail JOB DESCRIPTION Serve as a senior-level data engineer and ETL developer responsible for modernizing and sustaining data integration systems across the enterprise. This position also functions as the lead technical data architect on the OIS (Office of Information Services) architecture team, guiding the design and implementation of future-forward data platforms, pipelines, and integrations. The role is critical in bridging support for legacy SSIS-based systems with development of cutting-edge data engineering solutions, ensuring continuity, scalability, and innovation. RESPONSIBILITIES
Design, develop, and maintain both legacy ETL processes (SSIS) and modern data pipelines and orchestration mechanisms.
Architect scalable transactional and data warehouse solutions using relational, columnar, and cloud-based data stores.
Partner with cross-functional teams to define data architecture standards, integration patterns, and governance strategies.
Lead the technical evolution of enterprise data infrastructure in support of analytics, reporting, and system interoperability with transactional / OLTP data sources.
Work collaboratively with cross-functional teams in OIS, Office of Process Improvement and Data Applications (PIDA), and Office of Business Transformation (BT) to support and develop high functioning data integrations and related service levels for both transactional and analytical data platforms.
Provide mentorship and technical leadership across the data / analytics and application development teams.
Manage metadata, data lineage, and version/source control across data pipelines.
Work closely with agency IT Security and Data Governance programs to ensure data ETL follows agency and Commonwealth data governance and security standards and policies.
Serve as a point of escalation and guidance for performance tuning, troubleshooting, issue resolution and optimizations of legacy and modern data ecosystems.
QUALIFICATIONS
Minimum 8 years of combined experience in data engineering, data integration, or database architecture.
Proven record of architecting, developing and deploying enterprise-grade data solutions that span operational/transactional and analytical domains.
Expertise in logical and physical database design, data modeling (ER dimensional), and metadata management.
Proficiency in RDBMS design and development (SQL Server and Oracle required), including T-SQL, PL/SQL, and performance tuning; experience with database replication.
High-level proficiency with Python as a scripting and ETL automation language.
Demonstrated ability in developing and supporting ETL/ELT solutions across real-time and batch processing paradigms.
Expertise with legacy integration tools (SSIS) and modern ETL/data orchestration tools (Apache Airflow, Dagster, Airbyte, Perfect, etc.).
Experience with large-scale migrations/conversions; experience with Oracle-to-SQL Server conversion tools is a plus.
Familiarity with modern data warehouse ecosystems (dbt, Clickhouse, Snowflake, Databricks, Fivetran, etc.) in cloud environments.
Strong understanding of data warehousing, data lakes and data mesh patterns.
Excellent communication skills for working across diverse technical and non-technical teams.
Deep experience with SDLC and agile data engineering methodologies.
Ability to balance legacy system support with forward-looking architectural vision.
Experience integrating with RESTful APIs using JSON and implementing message-based architectures (e.g., RabbitMQ, Apache Kafka, Azure Service Bus) is a plus.
Prior experience in a technical leadership or enterprise architecture role preferred.
ABOUT US The Virginia Department of Environmental Quality (DEQ) is the primary environmental permitting agency in the Commonwealth of Virginia. It is responsible for administering laws and regulations related to air quality, water quality, water supply, renewable energy, and land protection. Through the dedication and work of over 800 employees across six regional offices and the Central Office in Richmond, DEQ issues permits, conducts monitoring and inspections, and enforces the law.