Lemongrass Consulting Inc
Senior Apache Kafka Engineer - Contractor
Lemongrass Consulting Inc, Chicago, Illinois, United States, 60290
Vacancy Name
Senior Apache Kafka Engineer - Contractor
Vacancy No VN1247
Business Unit Americas
Job Location Chicago, USA
Employment Type Full Time
Job Details and Responsibilities
We are seeking a highly skilled Senior Apache Kafka Engineer to lead the design, development, and operation of our enterprise-wide event streaming platform. This role requires deep expertise in Kafka as well as strong system analysis and design skills. The ideal candidate will have prior experience taking ownership of a complex, production-grade Kafka deployment and improving its scalability, resilience, and performance.
Key Responsibilities
Take full ownership of an existing, business-critical Kafka implementation: assess, stabilize, and evolve the architecture. Conduct comprehensive system analysis and design for event-driven and streaming data architectures. Design, build, and manage Kafka clusters across multiple environments (dev, staging, prod). Collaborate with engineering, product, and architecture teams to define and implement scalable streaming solutions. Design and optimize topic structures, partitioning strategies, and schema evolution approaches (e.g., using Avro). Build and maintain Kafka Connect pipelines and integration with external data sources and sinks. Ensure platform observability, reliability, security, and compliance with enterprise standards. Diagnose and resolve issues related to message delivery, latency, consumer lag, and broker performance. Contribute to documentation, operational runbooks, and internal training initiatives. Lead incident resolution, root cause analysis, and continuous improvement for Kafka infrastructure. Qualifications
Required Qualifications
5+ years of experience in software engineering or data platform roles, with 3+ years working directly with Apache Kafka in production. Demonstrated experience taking ownership of complex, large-scale Kafka environments. Strong system analysis and design capabilities, particularly in the context of distributed data systems. Deep understanding of Kafka architecture (brokers, partitions, replication, KRaft/ZooKeeper, ISR, etc.). Proficient in building Kafka-based data pipelines and real-time applications using Kafka Streams, ksqlDB, or similar tools. Programming expertise in Java, Scala, or Python. Familiarity with Kafka Connect, Schema Registry, and integrations with data systems like databases, Elasticsearch, cloud storage, and more. Hands-on experience with containerized platforms (Docker, Kubernetes) and infrastructure automation (Terraform, Helm). Experience managing Kafka in cloud environments (AWS MSK, Confluent Cloud, Azure Event Hubs, or self-managed). Knowledge of monitoring tools and alerting (Prometheus, Grafana, Control Center, Datadog, etc.). Preferred Qualifications
Confluent Certified Kafka Developer or Administrator. Experience leading Kafka modernization or re-architecture efforts. Background in data governance, event modeling, and schema versioning strategies. Familiarity with microservices architecture and event-driven design best practices. Soft Skills
Strong analytical thinking and problem-solving skills. Ability to communicate effectively with both technical and non-technical stakeholders. Ownership mindset and comfort navigating ambiguity in complex systems. Team leadership or mentoring experience is a plus. What We Offer
Competitive compensation and performance-based incentives. Flexible work environment and remote options. Access to training, certifications, and cutting-edge tools. Comprehensive benefits (medical, dental, vision, PTO, retirement).
Selected applicant will be subject to a background investigation, which will be conducted and the results of which will be used in compliance with applicable law.
Lemongrass Consulting is proud to be an Equal Opportunity and Affirmative Action employer. We do not discriminate on the basis of race, religion, color, national origin, religious creed, gender, sexual orientation, gender identity, gender expression, age, genetic information, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics.
About Lemongrass
Lemongrass (lemongrasscloud.com) is a global leader in SAP consulting, focused on helping organizations transform their business processes through innovative solutions and technologies. With a strong commitment to customer success, Lemongrass partners with companies to drive their digital transformation journeys, enabling them to unlock the full potential of their SAP investments.
We do this with our continuous innovation, automation, migration and operation, delivered on the world's most comprehensive cloud platforms - AWS, Azure and GCP and SAP Cloud ERP. We have been working with AWS and SAP since 2010 and we are a Premier Amazon Partner Network (APN) Consulting Partner. We are also a Microsoft Gold Partner, a Google Cloud Partner and an SAP Certified Silver Partner.
Our team is what makes Lemongrass exceptional and why we have the excellent reputation in the market that we enjoy today. At Lemongrass, you will work with the smartest and most motivated people in the business. We take pride in our culture of innovation and collaboration that drives us to deliver exceptional benefits to our clients every day.
Vacancy No VN1247
Business Unit Americas
Job Location Chicago, USA
Employment Type Full Time
Job Details and Responsibilities
We are seeking a highly skilled Senior Apache Kafka Engineer to lead the design, development, and operation of our enterprise-wide event streaming platform. This role requires deep expertise in Kafka as well as strong system analysis and design skills. The ideal candidate will have prior experience taking ownership of a complex, production-grade Kafka deployment and improving its scalability, resilience, and performance.
Key Responsibilities
Take full ownership of an existing, business-critical Kafka implementation: assess, stabilize, and evolve the architecture. Conduct comprehensive system analysis and design for event-driven and streaming data architectures. Design, build, and manage Kafka clusters across multiple environments (dev, staging, prod). Collaborate with engineering, product, and architecture teams to define and implement scalable streaming solutions. Design and optimize topic structures, partitioning strategies, and schema evolution approaches (e.g., using Avro). Build and maintain Kafka Connect pipelines and integration with external data sources and sinks. Ensure platform observability, reliability, security, and compliance with enterprise standards. Diagnose and resolve issues related to message delivery, latency, consumer lag, and broker performance. Contribute to documentation, operational runbooks, and internal training initiatives. Lead incident resolution, root cause analysis, and continuous improvement for Kafka infrastructure. Qualifications
Required Qualifications
5+ years of experience in software engineering or data platform roles, with 3+ years working directly with Apache Kafka in production. Demonstrated experience taking ownership of complex, large-scale Kafka environments. Strong system analysis and design capabilities, particularly in the context of distributed data systems. Deep understanding of Kafka architecture (brokers, partitions, replication, KRaft/ZooKeeper, ISR, etc.). Proficient in building Kafka-based data pipelines and real-time applications using Kafka Streams, ksqlDB, or similar tools. Programming expertise in Java, Scala, or Python. Familiarity with Kafka Connect, Schema Registry, and integrations with data systems like databases, Elasticsearch, cloud storage, and more. Hands-on experience with containerized platforms (Docker, Kubernetes) and infrastructure automation (Terraform, Helm). Experience managing Kafka in cloud environments (AWS MSK, Confluent Cloud, Azure Event Hubs, or self-managed). Knowledge of monitoring tools and alerting (Prometheus, Grafana, Control Center, Datadog, etc.). Preferred Qualifications
Confluent Certified Kafka Developer or Administrator. Experience leading Kafka modernization or re-architecture efforts. Background in data governance, event modeling, and schema versioning strategies. Familiarity with microservices architecture and event-driven design best practices. Soft Skills
Strong analytical thinking and problem-solving skills. Ability to communicate effectively with both technical and non-technical stakeholders. Ownership mindset and comfort navigating ambiguity in complex systems. Team leadership or mentoring experience is a plus. What We Offer
Competitive compensation and performance-based incentives. Flexible work environment and remote options. Access to training, certifications, and cutting-edge tools. Comprehensive benefits (medical, dental, vision, PTO, retirement).
Selected applicant will be subject to a background investigation, which will be conducted and the results of which will be used in compliance with applicable law.
Lemongrass Consulting is proud to be an Equal Opportunity and Affirmative Action employer. We do not discriminate on the basis of race, religion, color, national origin, religious creed, gender, sexual orientation, gender identity, gender expression, age, genetic information, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics.
About Lemongrass
Lemongrass (lemongrasscloud.com) is a global leader in SAP consulting, focused on helping organizations transform their business processes through innovative solutions and technologies. With a strong commitment to customer success, Lemongrass partners with companies to drive their digital transformation journeys, enabling them to unlock the full potential of their SAP investments.
We do this with our continuous innovation, automation, migration and operation, delivered on the world's most comprehensive cloud platforms - AWS, Azure and GCP and SAP Cloud ERP. We have been working with AWS and SAP since 2010 and we are a Premier Amazon Partner Network (APN) Consulting Partner. We are also a Microsoft Gold Partner, a Google Cloud Partner and an SAP Certified Silver Partner.
Our team is what makes Lemongrass exceptional and why we have the excellent reputation in the market that we enjoy today. At Lemongrass, you will work with the smartest and most motivated people in the business. We take pride in our culture of innovation and collaboration that drives us to deliver exceptional benefits to our clients every day.