Diverse Lynx
Overview
Job Description: Data Engineer Responsibilities
Senior Data Engineer with 6 to 8 Years of strong expertise in Python programming and Big Data technologies (Hadoop ecosystem) to design, develop, and optimize large-scale data pipelines. Design and implement scalable ETL pipelines using Python and Hadoop tools (Hive, Pig, HDFS). Automate data ingestion and transformation for structured and unstructured data. Develop distributed data processing workflows using Spark, MapReduce, and Hadoop. Optimize performance for high-volume datasets. Create efficient data models for analytics and reporting. Implement best practices for data partitioning and indexing in Hadoop. Monitor and tune Hadoop clusters for optimal performance. Ensure high availability and fault tolerance of data systems. Work closely with data scientists and analysts to enable predictive analytics and machine learning. Technical Skills
Strong Python programming for data processing and automation. Expertise in Hadoop ecosystem (HDFS, Hive, Pig, MapReduce). Experience with Spark for distributed computing. Proficiency in SQL and NoSQL databases. Soft Skills
Strong analytical and problem-solving skills. Experience with Agile/Scrum methodologies. Salary Range: $115,000-$130,000 a year Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.
#J-18808-Ljbffr
Job Description: Data Engineer Responsibilities
Senior Data Engineer with 6 to 8 Years of strong expertise in Python programming and Big Data technologies (Hadoop ecosystem) to design, develop, and optimize large-scale data pipelines. Design and implement scalable ETL pipelines using Python and Hadoop tools (Hive, Pig, HDFS). Automate data ingestion and transformation for structured and unstructured data. Develop distributed data processing workflows using Spark, MapReduce, and Hadoop. Optimize performance for high-volume datasets. Create efficient data models for analytics and reporting. Implement best practices for data partitioning and indexing in Hadoop. Monitor and tune Hadoop clusters for optimal performance. Ensure high availability and fault tolerance of data systems. Work closely with data scientists and analysts to enable predictive analytics and machine learning. Technical Skills
Strong Python programming for data processing and automation. Expertise in Hadoop ecosystem (HDFS, Hive, Pig, MapReduce). Experience with Spark for distributed computing. Proficiency in SQL and NoSQL databases. Soft Skills
Strong analytical and problem-solving skills. Experience with Agile/Scrum methodologies. Salary Range: $115,000-$130,000 a year Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.
#J-18808-Ljbffr