Collabera
Job Title
Hadoop Data Engineer
Location:
Chicago, IL
Work Arrangement:
Onsite
Client Industry:
Financial Services
Duration:
12 Months Contract
Schedule:
Standard Hours (Monday - Friday, 8 AM to 5 PM)
Seniority level:
Entry level
Employment type:
Contract
Job function:
Information Technology
Day-to-Day Responsibilities
Design, develop, and maintain batch and near‑real‑time data pipelines using Spark Structured Streaming, MapReduce, and other Big Data frameworks.
Ingest data from multiple sources such as message queues (Kafka), file shares, REST APIs, and relational databases.
Transform, clean, and validate data in HDFS, Hive, Impala, or Spark SQL.
Convert and manage data in formats like JSON, CSV, XML to support downstream analytics.
Perform data validation, profiling, and analysis to identify anomalies and ensure data integrity.
Troubleshoot issues in data pipelines, SQL jobs, or Spark applications, including slow‑running jobs or failures.
Must Haves
Strong SQL skills – MySQL, Hive, Impala, or Spark SQL.
Experience ingesting data from message queues, file shares, REST APIs, relational databases, and handling formats such as JSON, CSV, XML.
Experience working with Spark Structured steaming.
Experience with Hadoop/Big Data and distributed systems.
Hands‑on programming experience in at least one of Scala, Python, or PHP.
Working knowledge of Spark, Sqoop, Kafka, MapReduce, NoSQL databases (e.g., HBase, Solr, CDP, HDP, Cloudera, Hortonworks), Elasticsearch, Kibana.
Plusses
Prior bank experience.
Experience with ServiceNow.
Compensation Hourly Rate:
$60 – $65 per hour
This reflects base compensation and may vary based on candidate qualifications.
Benefits The Company offers the following benefits for this position, subject to applicable eligibility requirements: medical insurance, dental insurance, vision insurance, 401(k) retirement plan, life insurance, long‑term disability insurance, short‑term disability insurance, paid parking/public transportation, paid time off, paid sick and safe time, hours of paid vacation time, weeks of paid parental leave, and paid holidays annually – as applicable.
At Collabera, we build careers by providing opportunities to work with top organizations, cutting‑edge technologies, and dynamic teams. Join a culture that thrives on innovation, collaboration, and a commitment to excellence.
#J-18808-Ljbffr
Location:
Chicago, IL
Work Arrangement:
Onsite
Client Industry:
Financial Services
Duration:
12 Months Contract
Schedule:
Standard Hours (Monday - Friday, 8 AM to 5 PM)
Seniority level:
Entry level
Employment type:
Contract
Job function:
Information Technology
Day-to-Day Responsibilities
Design, develop, and maintain batch and near‑real‑time data pipelines using Spark Structured Streaming, MapReduce, and other Big Data frameworks.
Ingest data from multiple sources such as message queues (Kafka), file shares, REST APIs, and relational databases.
Transform, clean, and validate data in HDFS, Hive, Impala, or Spark SQL.
Convert and manage data in formats like JSON, CSV, XML to support downstream analytics.
Perform data validation, profiling, and analysis to identify anomalies and ensure data integrity.
Troubleshoot issues in data pipelines, SQL jobs, or Spark applications, including slow‑running jobs or failures.
Must Haves
Strong SQL skills – MySQL, Hive, Impala, or Spark SQL.
Experience ingesting data from message queues, file shares, REST APIs, relational databases, and handling formats such as JSON, CSV, XML.
Experience working with Spark Structured steaming.
Experience with Hadoop/Big Data and distributed systems.
Hands‑on programming experience in at least one of Scala, Python, or PHP.
Working knowledge of Spark, Sqoop, Kafka, MapReduce, NoSQL databases (e.g., HBase, Solr, CDP, HDP, Cloudera, Hortonworks), Elasticsearch, Kibana.
Plusses
Prior bank experience.
Experience with ServiceNow.
Compensation Hourly Rate:
$60 – $65 per hour
This reflects base compensation and may vary based on candidate qualifications.
Benefits The Company offers the following benefits for this position, subject to applicable eligibility requirements: medical insurance, dental insurance, vision insurance, 401(k) retirement plan, life insurance, long‑term disability insurance, short‑term disability insurance, paid parking/public transportation, paid time off, paid sick and safe time, hours of paid vacation time, weeks of paid parental leave, and paid holidays annually – as applicable.
At Collabera, we build careers by providing opportunities to work with top organizations, cutting‑edge technologies, and dynamic teams. Join a culture that thrives on innovation, collaboration, and a commitment to excellence.
#J-18808-Ljbffr