HEITMEYER CONSULTING INC
To Apply for this Job Click Here
Heitmeyer Consulting has a banking client that has a need within their Chief Data Office for a strong Data Engineer to specialize in building and optimizing high-performance, real-time data pipelines. This role is central to leveraging the power of Apache Kafka for event streaming and Apache Flink for complex, stateful stream processing and analytics. The ideal candidate will transform raw, high-velocity data into actionable, low-latency insights that drive core business functionality, working within our AWS-based data ecosystem leveraging S3 for storage. Role must be based in Dallas, TX or Tulsa, OK.
Top Required Skills:
Require extensive experience as a
Data Engineer
building and maintaining production-grade data pipelines with a focus on real-time systems.
In-depth experience with Apache Kafka along with hands‑on experience with Apache Flink, including understanding of state management, fault tolerance and time semantics.
Significant cloud background
with strong expertise in AWS cloud services related to data engineering (S3, MSK, Glue, Athena, EMR, DynamoDB, etc.).
Proficiency in
programming/scripting languages such as Python, Java, or Scala .
Familiar with
modern data stack and big data technologies
(Spark, Airflow, Iceberg, SQL) along with understanding of distributed systems and processing real-time data at scale.
Excellent problem-solving skills and the ability to troubleshoot complex issues in distributed systems.
Nice-to-have:
Background within financial services would be preferred but not required.
Key Responsibilities
Design, develop, and deploy robust, high-throughput, low-latency streaming applications
using Apache Flink and Java/Scala/Python APIs.
Architect and optimize Kafka-based event‑driven systems , including topic design, schema enforcement via Schema Registry, and integration using Kafka Connect.
Lead data transformation
through implementation of complex business logic, stateful processing, windowing, and data enrichment within Flink jobs to transform raw Kafka streams into analytics-ready datasets, ensuring exactly‑once semantics and fault tolerance.
Ensure cloud integration
with storage and manage processed data efficiently in AWS S3 and potentially using an open format like Apache Iceberg, connecting seamlessly with downstream analytics tools (e.g., Athena).
Optimize performance and observability
– tune Flink and Kafka configurations for maximum performance, efficiency, and scalability, implementing robust monitoring and alerting for production environments.
Partner closely
with data scientists, software engineers, and business stakeholders to translate functional requirements into reliable data processing solutions.
Champion
software engineering best practices , including automated testing, code review, CI/CD pipelines, and documentation of data lineage and logic.
Heitmeyer Consulting is an equal opportunity employer, and we encourage all qualified candidates to apply. Qualified applicants will be considered without regard to minority status, gender, disability, veteran status or any other characteristic protected by law.
To Apply for this Job Click Here #J-18808-Ljbffr
Top Required Skills:
Require extensive experience as a
Data Engineer
building and maintaining production-grade data pipelines with a focus on real-time systems.
In-depth experience with Apache Kafka along with hands‑on experience with Apache Flink, including understanding of state management, fault tolerance and time semantics.
Significant cloud background
with strong expertise in AWS cloud services related to data engineering (S3, MSK, Glue, Athena, EMR, DynamoDB, etc.).
Proficiency in
programming/scripting languages such as Python, Java, or Scala .
Familiar with
modern data stack and big data technologies
(Spark, Airflow, Iceberg, SQL) along with understanding of distributed systems and processing real-time data at scale.
Excellent problem-solving skills and the ability to troubleshoot complex issues in distributed systems.
Nice-to-have:
Background within financial services would be preferred but not required.
Key Responsibilities
Design, develop, and deploy robust, high-throughput, low-latency streaming applications
using Apache Flink and Java/Scala/Python APIs.
Architect and optimize Kafka-based event‑driven systems , including topic design, schema enforcement via Schema Registry, and integration using Kafka Connect.
Lead data transformation
through implementation of complex business logic, stateful processing, windowing, and data enrichment within Flink jobs to transform raw Kafka streams into analytics-ready datasets, ensuring exactly‑once semantics and fault tolerance.
Ensure cloud integration
with storage and manage processed data efficiently in AWS S3 and potentially using an open format like Apache Iceberg, connecting seamlessly with downstream analytics tools (e.g., Athena).
Optimize performance and observability
– tune Flink and Kafka configurations for maximum performance, efficiency, and scalability, implementing robust monitoring and alerting for production environments.
Partner closely
with data scientists, software engineers, and business stakeholders to translate functional requirements into reliable data processing solutions.
Champion
software engineering best practices , including automated testing, code review, CI/CD pipelines, and documentation of data lineage and logic.
Heitmeyer Consulting is an equal opportunity employer, and we encourage all qualified candidates to apply. Qualified applicants will be considered without regard to minority status, gender, disability, veteran status or any other characteristic protected by law.
To Apply for this Job Click Here #J-18808-Ljbffr