Macpower Digital Assets Edge
Job Overview:
We are seeking an experienced
Kafka Engineer
with expertise in
Confluent Kafka, Java/Scala, and distributed systems . The ideal candidate should be skilled in designing
scalable, fault-tolerant Kafka-based data pipelines , troubleshooting messaging issues, and optimizing performance. A strong background in
cloud deployments, microservices, and Agile development
with an
utomate-first
approach is essential. Responsibilities: Identify and resolve
Kafka messaging issues
within a justified timeframe. Collaborate with business and IT teams to understand business problems and design, implement, and deliver appropriate solutions using
gile methodology
within a larger program. Work independently to implement solutions across
multiple environments
(DEV, QA, UAT, PROD). Provide
technical direction, guidance, and code reviews
for other engineers working on the same project. dminister
distributed Kafka clusters
in DEV, QA, UAT, and PROD environments and troubleshoot
performance issues . Implement and debug
subsystems, microservices, and components . Follow an
utomate-first/automate-everything
philosophy. Demonstrate hands-on experience with
programming languages
relevant to the role. Key Skills & Expertise:
Deep understanding of Confluent Kafka
- Proficient in Kafka concepts, including
producers, consumers, topics, partitions, brokers, and replication mechanisms . Programming proficiency
- Expertise in
Java or Scala , with potential Python usage depending on the project. System design and architecture
- Ability to design
robust, scalable Kafka-based data pipelines
considering
data throughput, fault tolerance, and latency . Data management skills
- Knowledge of
data serialization formats
such as
JSON, Avro, and Protobuf , and
schema evolution management . Kafka Streams API (optional)
- Familiarity with
Kafka Streams
for
real-time data processing
within the Kafka ecosystem. Monitoring & troubleshooting
- Experience with
Kafka cluster health monitoring , identifying
performance bottlenecks , and troubleshooting issues. Cloud integration
- Experience deploying and managing Kafka on
WS, Azure, or GCP . Understanding of distributed systems concepts . Must-Have Qualifications:
8-12 years of experience
in software engineering. Kafka expertise
- Deep knowledge of
Kafka producers, consumers, topics, partitions, brokers, and replication . Programming proficiency
- Strong in
Java or Scala , with potential Python usage. System design & architecture
- Experience in designing
high-throughput, scalable Kafka pipelines . Cloud & DevOps
- Experience deploying
Kafka on AWS, Azure, or GCP . Monitoring & troubleshooting
- Familiarity with
Kafka cluster health monitoring and performance tuning .
We are seeking an experienced
Kafka Engineer
with expertise in
Confluent Kafka, Java/Scala, and distributed systems . The ideal candidate should be skilled in designing
scalable, fault-tolerant Kafka-based data pipelines , troubleshooting messaging issues, and optimizing performance. A strong background in
cloud deployments, microservices, and Agile development
with an
utomate-first
approach is essential. Responsibilities: Identify and resolve
Kafka messaging issues
within a justified timeframe. Collaborate with business and IT teams to understand business problems and design, implement, and deliver appropriate solutions using
gile methodology
within a larger program. Work independently to implement solutions across
multiple environments
(DEV, QA, UAT, PROD). Provide
technical direction, guidance, and code reviews
for other engineers working on the same project. dminister
distributed Kafka clusters
in DEV, QA, UAT, and PROD environments and troubleshoot
performance issues . Implement and debug
subsystems, microservices, and components . Follow an
utomate-first/automate-everything
philosophy. Demonstrate hands-on experience with
programming languages
relevant to the role. Key Skills & Expertise:
Deep understanding of Confluent Kafka
- Proficient in Kafka concepts, including
producers, consumers, topics, partitions, brokers, and replication mechanisms . Programming proficiency
- Expertise in
Java or Scala , with potential Python usage depending on the project. System design and architecture
- Ability to design
robust, scalable Kafka-based data pipelines
considering
data throughput, fault tolerance, and latency . Data management skills
- Knowledge of
data serialization formats
such as
JSON, Avro, and Protobuf , and
schema evolution management . Kafka Streams API (optional)
- Familiarity with
Kafka Streams
for
real-time data processing
within the Kafka ecosystem. Monitoring & troubleshooting
- Experience with
Kafka cluster health monitoring , identifying
performance bottlenecks , and troubleshooting issues. Cloud integration
- Experience deploying and managing Kafka on
WS, Azure, or GCP . Understanding of distributed systems concepts . Must-Have Qualifications:
8-12 years of experience
in software engineering. Kafka expertise
- Deep knowledge of
Kafka producers, consumers, topics, partitions, brokers, and replication . Programming proficiency
- Strong in
Java or Scala , with potential Python usage. System design & architecture
- Experience in designing
high-throughput, scalable Kafka pipelines . Cloud & DevOps
- Experience deploying
Kafka on AWS, Azure, or GCP . Monitoring & troubleshooting
- Familiarity with
Kafka cluster health monitoring and performance tuning .