Pyramid Consulting, Inc
Lead ETL Quality Engineer
Pyramid Consulting, Inc, Downers Grove, Illinois, United States, 60516
Immediate need for a talented Lead ETL Quality Engineer. This is a 12+months contract opportunity with long-term potential and is located in U.S(Remote). Please review the job description below and contact me ASAP if you are interested.
Job ID : 25-86296
Pay Range : $75 - $80 / hour. Employee benefits include, but are not limited to, health insurance (medical, dental, vision), 401(k) plan, and paid sick leave (depending on work location).
Key Responsibilities
Design and execute test strategies for ETL pipelines and data warehouse processes.
Validate data extraction from multiple source systems (databases, APIs, flat files).
Verify transformations align with business rules and data mapping specifications.
Confirm data loading into target systems (data warehouse, data lake, or reporting platform) is accurate and complete.
Build automated testing frameworks using SQL, Python, and Databricks QA tooling.
QA data pipelines, workloads, and transformations specifically within Databricks.
Apply Databricks QA tools such as DQX, Client Live Tables (DLT) Expectations, and Cuallee (or similar) to validate data quality.
Create regression, functional, and performance test suites to support continuous integration (CI / CD).
Monitor data quality, identify issues (duplicates, missing data, incorrect aggregations), and report defects.
Collaborate with data engineers to troubleshoot ETL bugs and optimize data pipeline performance.
Document test cases, test results, and maintain traceability to business requirements.
Key Requirements and Technology Experience
Key Skills : ETL, Quality Engineering, Databricks, DQX, Client Live Tables (DLT) Expectations, and Cuallee (or similar).
Bachelor’s degree in computer science, Information Systems, Engineering, or related field.
3+ years of experience in ETL, data warehouse, or data pipeline testing.
Strong hands‑on experience with SQL for data validation and reconciliation.
Hands‑on proficiency with Databricks QA tooling (DQX, DLT Expectations, Cuallee, or similar).
Experience QA’ing data pipelines, workloads, and transformations in Databricks.
Familiarity with ETL tools (e.g., Informatica, Talend, SSIS, DataStage, dbt).
Knowledge of cloud data warehouses (e.g., Client, Redshift, BigQuery).
Proficiency in Python, Java, or shell scripting for test automation.
Experience with CI / CD pipelines (e.g., Jenkins, GitLab, Azure DevOps) is a plus.
Strong analytical and problem‑solving skills with attention to detail.
Our client is a leading IT Consulting Industry, and we are currently interviewing to fill this and other similar contract positions. If you are interested in this position, please apply online for immediate consideration.
Pyramid Consulting, Inc. provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, colour, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.
By applying to our jobs you agree to receive calls, AI‑generated calls, text messages, or emails from Pyramid Consulting, Inc. and its affiliates, and contracted partners. Frequency varies for text messages. Message and data rates may apply. Carriers are not liable for delayed or undelivered messages. You can reply STOP to cancel and HELP for help. You can access our privacy policy here .
#J-18808-Ljbffr
Job ID : 25-86296
Pay Range : $75 - $80 / hour. Employee benefits include, but are not limited to, health insurance (medical, dental, vision), 401(k) plan, and paid sick leave (depending on work location).
Key Responsibilities
Design and execute test strategies for ETL pipelines and data warehouse processes.
Validate data extraction from multiple source systems (databases, APIs, flat files).
Verify transformations align with business rules and data mapping specifications.
Confirm data loading into target systems (data warehouse, data lake, or reporting platform) is accurate and complete.
Build automated testing frameworks using SQL, Python, and Databricks QA tooling.
QA data pipelines, workloads, and transformations specifically within Databricks.
Apply Databricks QA tools such as DQX, Client Live Tables (DLT) Expectations, and Cuallee (or similar) to validate data quality.
Create regression, functional, and performance test suites to support continuous integration (CI / CD).
Monitor data quality, identify issues (duplicates, missing data, incorrect aggregations), and report defects.
Collaborate with data engineers to troubleshoot ETL bugs and optimize data pipeline performance.
Document test cases, test results, and maintain traceability to business requirements.
Key Requirements and Technology Experience
Key Skills : ETL, Quality Engineering, Databricks, DQX, Client Live Tables (DLT) Expectations, and Cuallee (or similar).
Bachelor’s degree in computer science, Information Systems, Engineering, or related field.
3+ years of experience in ETL, data warehouse, or data pipeline testing.
Strong hands‑on experience with SQL for data validation and reconciliation.
Hands‑on proficiency with Databricks QA tooling (DQX, DLT Expectations, Cuallee, or similar).
Experience QA’ing data pipelines, workloads, and transformations in Databricks.
Familiarity with ETL tools (e.g., Informatica, Talend, SSIS, DataStage, dbt).
Knowledge of cloud data warehouses (e.g., Client, Redshift, BigQuery).
Proficiency in Python, Java, or shell scripting for test automation.
Experience with CI / CD pipelines (e.g., Jenkins, GitLab, Azure DevOps) is a plus.
Strong analytical and problem‑solving skills with attention to detail.
Our client is a leading IT Consulting Industry, and we are currently interviewing to fill this and other similar contract positions. If you are interested in this position, please apply online for immediate consideration.
Pyramid Consulting, Inc. provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, colour, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.
By applying to our jobs you agree to receive calls, AI‑generated calls, text messages, or emails from Pyramid Consulting, Inc. and its affiliates, and contracted partners. Frequency varies for text messages. Message and data rates may apply. Carriers are not liable for delayed or undelivered messages. You can reply STOP to cancel and HELP for help. You can access our privacy policy here .
#J-18808-Ljbffr