Diverse Lynx
Role Description:
• Lead the design, development, and deployment of PySpark-based big data solutions.• Architect and optimize ETL pipelines for structured and unstructured data.• Collaborate with Client, data engineers, data scientists, and business teams to understand requirements and provide scalable solutions.• Optimize Spark performance through partitioning, caching, and tuning.• Implement best practices in data engineering (CI/CD, version control, unit testing).• Work with cloud platforms like AWS • Ensure data security, governance, and compliance.• Mentor junior developers and review code for best practices and efficiency.
Competencies:
Digital : Amazon Web Service(AWS) Cloud Computing, Digital : PySpark
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.
• Lead the design, development, and deployment of PySpark-based big data solutions.• Architect and optimize ETL pipelines for structured and unstructured data.• Collaborate with Client, data engineers, data scientists, and business teams to understand requirements and provide scalable solutions.• Optimize Spark performance through partitioning, caching, and tuning.• Implement best practices in data engineering (CI/CD, version control, unit testing).• Work with cloud platforms like AWS • Ensure data security, governance, and compliance.• Mentor junior developers and review code for best practices and efficiency.
Competencies:
Digital : Amazon Web Service(AWS) Cloud Computing, Digital : PySpark
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.