Insight Global
Job Description
Insight Global is seeking a Data Engineer for a contract to hire opportunity at one of our trading and commodities clients based in Chicago, IL. This role ideally sits local to the Chicago, IL area, but if a candidate is available to relocate to Chicago, IL that would be considered.
This individual will be responsible for assisting a team in building out the entirety of the data product lifecycle within the modernization and migration of the clients databricks platform and azure cloud. The candidate will be responsible for the build out, maintain scalable data pipelines and cloud (Azure) based infrastructure . Additionally, this candidate will be responsible for ensuring reliable data flow, optimize database performance and support key and critical data driven decision making within the organization. Ability to write efficient, reusable, and testable python scripting/code for data ingestion, transformation, and validation will be a critical function of this role.
Other responsibilities will include:
Proven experience with Databricks and Azure cloud services (e.g., Data Lake, Synapse, Data Factory).
Strong proficiency in Python for data engineering tasks
Solid understanding of ETL/ELT processes, data modeling, and distributed computing
Experience with SQL and working with large-scale relational and non-relational databases
Familiarity with CI/CD pipelines and version control Github for integration
We are a company committed to creating inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity employer that believes everyone matters. Qualified candidates will receive consideration for employment opportunities without regard to race, religion, sex, age, marital status, national origin, sexual orientation, citizenship status, disability, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to Human Resources Request Form (https://airtable.com/app21VjYyxLDIX0ez/shrOg4IQS1J6dRiMo) . The EEOC "Know Your Rights" Poster is available here (https://www.eeoc.gov/sites/default/files/2023-06/22-088_EEOC_KnowYourRights6.12ScreenRdr.pdf) .
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .
Skills and Requirements
3-5 years of experience as a Data Engineer migrating from on Prem to Databricks
Experience with Python and Azure
3-5 years of experience working within the Data Product Lifecycle
3+ years of experience in migrating legacy systems to SQL
Ability to work in a fast paced, highly visible environment null
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion,sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military oruniformed service member status, or any other status or characteristic protected by applicable laws, regulations, andordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to HR@insightglobal.com.
Insight Global is seeking a Data Engineer for a contract to hire opportunity at one of our trading and commodities clients based in Chicago, IL. This role ideally sits local to the Chicago, IL area, but if a candidate is available to relocate to Chicago, IL that would be considered.
This individual will be responsible for assisting a team in building out the entirety of the data product lifecycle within the modernization and migration of the clients databricks platform and azure cloud. The candidate will be responsible for the build out, maintain scalable data pipelines and cloud (Azure) based infrastructure . Additionally, this candidate will be responsible for ensuring reliable data flow, optimize database performance and support key and critical data driven decision making within the organization. Ability to write efficient, reusable, and testable python scripting/code for data ingestion, transformation, and validation will be a critical function of this role.
Other responsibilities will include:
Proven experience with Databricks and Azure cloud services (e.g., Data Lake, Synapse, Data Factory).
Strong proficiency in Python for data engineering tasks
Solid understanding of ETL/ELT processes, data modeling, and distributed computing
Experience with SQL and working with large-scale relational and non-relational databases
Familiarity with CI/CD pipelines and version control Github for integration
We are a company committed to creating inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity employer that believes everyone matters. Qualified candidates will receive consideration for employment opportunities without regard to race, religion, sex, age, marital status, national origin, sexual orientation, citizenship status, disability, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to Human Resources Request Form (https://airtable.com/app21VjYyxLDIX0ez/shrOg4IQS1J6dRiMo) . The EEOC "Know Your Rights" Poster is available here (https://www.eeoc.gov/sites/default/files/2023-06/22-088_EEOC_KnowYourRights6.12ScreenRdr.pdf) .
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .
Skills and Requirements
3-5 years of experience as a Data Engineer migrating from on Prem to Databricks
Experience with Python and Azure
3-5 years of experience working within the Data Product Lifecycle
3+ years of experience in migrating legacy systems to SQL
Ability to work in a fast paced, highly visible environment null
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion,sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military oruniformed service member status, or any other status or characteristic protected by applicable laws, regulations, andordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to HR@insightglobal.com.