Diverse Lynx
Role Description: Data Pipeline Testing Designing and executing test cases for ETL (Extract, Transform, Load) workflows and data validation, focusing on both batch and streaming data. Data Validation Validating data ingestion, transformation, and loading processes within Kafka, ensuring data quality, consistency, and completeness across systems. Kafka Expertise Hands-on experience with Apache Kafka for data streaming validation, including working with producers, consumers, and understanding Kafkas architecture and concepts. Scripting and Automation Proficiency in Python or other scripting languages for automating testing of data pipelines and maintaining test scripts. Issue Resolution Collaborating with developers to identify, analyze, and resolve data issues, performing root cause analysis, and proposing solutions. Performance Testing Evaluating the performance and scalability of Kafka-based systems, ensuring they can handle the expected data volume and throughput. AgileScrum Understanding and working within AgileScrum methodologies and participating in testing activities within the development lifecycle. Data Warehousing Familiarity with data warehousing concepts and cloud data platforms is often a plus. Essential Skills: Apache Kafka Deep understanding of Kafkas architecture, components, and functionality. SQL Experience with SQL for data validation and debugging. ETL Experience with ETL processes and data warehousing concepts. Test Automation Experience in developing and maintaining automated tests for data pipelines. Data Analysis Ability to analyze data, identify issues, and propose solutions. Communication Strong communication and collaboration skills are essential for working with development teams.Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.