Logo
Highmark Health

Lead Big Data Engineer

Highmark Health, Santa Fe, New Mexico, us, 87503

Save Job

Company:

Highmark Inc. Job Summary:

We are seeking a passionate and experienced Lead Big Data Engineer who will play a crucial role in designing and developing robust data solutions for our enterprise. In this collaborative position, you will engage with business, analytics, and IT stakeholders to create, maintain, and improve data ingestion pipelines, transformations, and delivery programs, enabling effective data access for operational and analytical purposes. Key Responsibilities: Collaborate with cross-functional teams to design, build, and maintain exceptional data solutions across various environments, including traditional data warehouses, Big Data, and cloud platforms. Create high-performance data systems tailored for operational and analytical applications. Integrate internal and external platforms to ensure aligned data sourcing and flow. Engage with business stakeholders to implement and support operational and analytical platforms, ensuring smooth integration of purchased products and maintained enterprise data. Develop, test, implement, and maintain scalable frameworks for effective data engineering services across multiple systems. Contribute to metadata management, data quality management, and the application of business rules, while ensuring compliance with security and governance standards. Develop and optimize MLOps processes for cloud and Big Data environments. Mentor Junior and Intermediate team members, providing guidance on best practices. Establish standards for effective data ingestion, transformation, and delivery to meet analytical needs. Stay updated with the latest Big Data technologies and recommend appropriate tools for current and future projects. Education: Required: Bachelor's Degree in Software Engineering, Information Systems, Computer Science, Data Science, or a related field. Preferred: Master's Degree in the above-mentioned fields. Experience: Required: Minimum of 5 years in data platform development, data engineering, software development, or data science. Minimum of 3 years in Big Data or cloud data platforms. Preferred: 3 years experience in the healthcare industry. 3 years in Data Warehousing. 3 years in Database Administration. Certifications: Required: None. Preferred: Cloud certification (GCP, Azure, AWS). Skills: Proficient in SQL and Data Warehousing. Excellent problem-solving and analytical skills. Strong communication abilities. Experience with Spark, Python, or related tools. Familiarity with cloud technologies. Travel Requirement:

0% - 25% Location:

Office-based, with no physical work site required. This role includes various responsibilities and opportunities for growth. If you are ready to contribute to innovative data solutions, we encourage you to apply!