Arbitration Forums
DEPARTMENT:
Data Insights and Innovation
JOB TITLE:
GenAI Engineering Lead
JOB CODE:
GAEL
REPORTS TO:
Director, Head of Data Insights and Innovation
JOB PURPOSE:
The GenAI Engineering Lead is responsible for designing, developing, optimizing, and testing prompts for generative AI to guide large language models in producing accurate, relevant, and high-quality outputs across various business applications. The ideal candidate will have a strong background in AI, natural language processing, and deep expertise in Large Language Models and prompt engineering. The GenAI Engineering Lead will partner with stakeholders to drive business value to Arbitration Forums and our members through AI solutions.
DEPARTMENTAL EXPECTATION OF EMPLOYEE Adheres to AF Policy and Procedures and the AF Leadership Brand Acts as a role model within and outside AF. Performs duties as workload necessitates. Maintains a positive and respectful attitude. Communicates regularly with supervisor about Department issues. Demonstrates flexible and efficient time management and ability to prioritize workload. Consistently reports to work on time, prepared to perform duties of position. Meets Department productivity standards. ESSENTIAL DUTIES AND RESPONSIBILITIES
Prompt Design and Optimization:
Design, develop, and refine prompts for LLMs to ensure high-quality outputs for specific use cases. Develop and refine prompts for various generative AI models to achieve desired outputs. Evaluate and improve prompt performance using both qualitative and quantitative metrics. Conduct experiments to test and improve prompt effectiveness and response quality. Conduct A/B testing of prompt variants and analyze model behaviors. Stay up to date with advancements in LLM capabilities and prompt engineering best practices. Collaborate with product managers, data scientists, ML engineers, and business stakeholders to integrate GenAI into business processes, products, or workflows. Create documentation and reusable prompt libraries for internal and external use. Develop safety filters and guide the ethical use of prompts to prevent biased or harmful responses.
AI Governance and Security:
Work with the data governance lead to establish and enforce data governance standards, guidelines, processes, and best practices to ensure data quality and integrity as they apply to GenAI and NLP. Ensure that data security protocols are followed in the definition and implementation of AI solutions in accordance with regulatory requirements and company policies.
Collaboration and Strategy:
Work closely with IT, products, data engineers, data analysts, data scientists, and business stakeholders to understand data requirements and implement solutions. Provide technical leadership and mentorship to junior data team members. Provide training and support to team members on effective prompt engineering strategies. Stay updated on emerging data technologies and best practices, making recommendations for continuous improvement.
ADDITIONAL DUTIES AND RESPONSIBILITIES
Support model observability efforts to ensure adherence to company policies and enforce governance standards. Other duties as assigned by manager or project needs. QUALIFICATIONS
Experience, Education and Licensure
Bachelor's or Master's degree in Computer Science, Information Systems, Data Science, Linguistics, or a related field. Functional knowledge of composable products with a high level of configurability. Minimum of 7 years of experience in NLP, GenAI, LLMs, data governance, data science, or a related role. Technical Skills:
Proficiency prompt engineering for generative AI models (GPT-4, DALL-E, etc.). Strong understanding of natural language processing concepts and techniques. Proficiency in programming languages such as Python and familiarity with AI frameworks (e.g., TensorFlow, PyTorch). Working knowledge of cloud services (i.e., MS Azure, AWS, Google Cloud). Experience with AI tools, such as MS Azure ML, Snowflake CortexAI, Dataiku. Strong knowledge of data governance, data security, and compliance practices. Familiarity with data visualization and reporting tools (e.g., Webfocus, Power BI). Proficiency in programming languages such as Python, R, or SQL. Soft Skills:
Excellent analytical and problem-solving abilities. Strong communication and interpersonal skills to collaborate with cross-functional teams. Ability to lead projects and mentor junior staff. Auto Insurance claims industry experience preferred.
AMERICANS WITH DISABILITY SPECIFICATIONS
PHYSICAL DEMANDS
The physical demands described here are representative of those that must be met by an employee to successfully perform the essential functions of this job. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.
While performing the duties of this job, the employee is occasionally required to stand; walk; sit; use hands to finger, handle, or feel objects, tools, or controls; reach with hands and arms; climb stairs; balance; stoop, kneel, crouch or crawl; talk or hear; taste or smell. The employee must occasionally lift and/or move up to 25 pounds. Specific vision abilities required by the job include close vision, distance vision, color vision, peripheral vision, depth perception, and the ability to adjust focus.
WORK ENVIRONMENT
Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.
AF is a remote working environment.
Data Insights and Innovation
JOB TITLE:
GenAI Engineering Lead
JOB CODE:
GAEL
REPORTS TO:
Director, Head of Data Insights and Innovation
JOB PURPOSE:
The GenAI Engineering Lead is responsible for designing, developing, optimizing, and testing prompts for generative AI to guide large language models in producing accurate, relevant, and high-quality outputs across various business applications. The ideal candidate will have a strong background in AI, natural language processing, and deep expertise in Large Language Models and prompt engineering. The GenAI Engineering Lead will partner with stakeholders to drive business value to Arbitration Forums and our members through AI solutions.
DEPARTMENTAL EXPECTATION OF EMPLOYEE Adheres to AF Policy and Procedures and the AF Leadership Brand Acts as a role model within and outside AF. Performs duties as workload necessitates. Maintains a positive and respectful attitude. Communicates regularly with supervisor about Department issues. Demonstrates flexible and efficient time management and ability to prioritize workload. Consistently reports to work on time, prepared to perform duties of position. Meets Department productivity standards. ESSENTIAL DUTIES AND RESPONSIBILITIES
Prompt Design and Optimization:
Design, develop, and refine prompts for LLMs to ensure high-quality outputs for specific use cases. Develop and refine prompts for various generative AI models to achieve desired outputs. Evaluate and improve prompt performance using both qualitative and quantitative metrics. Conduct experiments to test and improve prompt effectiveness and response quality. Conduct A/B testing of prompt variants and analyze model behaviors. Stay up to date with advancements in LLM capabilities and prompt engineering best practices. Collaborate with product managers, data scientists, ML engineers, and business stakeholders to integrate GenAI into business processes, products, or workflows. Create documentation and reusable prompt libraries for internal and external use. Develop safety filters and guide the ethical use of prompts to prevent biased or harmful responses.
AI Governance and Security:
Work with the data governance lead to establish and enforce data governance standards, guidelines, processes, and best practices to ensure data quality and integrity as they apply to GenAI and NLP. Ensure that data security protocols are followed in the definition and implementation of AI solutions in accordance with regulatory requirements and company policies.
Collaboration and Strategy:
Work closely with IT, products, data engineers, data analysts, data scientists, and business stakeholders to understand data requirements and implement solutions. Provide technical leadership and mentorship to junior data team members. Provide training and support to team members on effective prompt engineering strategies. Stay updated on emerging data technologies and best practices, making recommendations for continuous improvement.
ADDITIONAL DUTIES AND RESPONSIBILITIES
Support model observability efforts to ensure adherence to company policies and enforce governance standards. Other duties as assigned by manager or project needs. QUALIFICATIONS
Experience, Education and Licensure
Bachelor's or Master's degree in Computer Science, Information Systems, Data Science, Linguistics, or a related field. Functional knowledge of composable products with a high level of configurability. Minimum of 7 years of experience in NLP, GenAI, LLMs, data governance, data science, or a related role. Technical Skills:
Proficiency prompt engineering for generative AI models (GPT-4, DALL-E, etc.). Strong understanding of natural language processing concepts and techniques. Proficiency in programming languages such as Python and familiarity with AI frameworks (e.g., TensorFlow, PyTorch). Working knowledge of cloud services (i.e., MS Azure, AWS, Google Cloud). Experience with AI tools, such as MS Azure ML, Snowflake CortexAI, Dataiku. Strong knowledge of data governance, data security, and compliance practices. Familiarity with data visualization and reporting tools (e.g., Webfocus, Power BI). Proficiency in programming languages such as Python, R, or SQL. Soft Skills:
Excellent analytical and problem-solving abilities. Strong communication and interpersonal skills to collaborate with cross-functional teams. Ability to lead projects and mentor junior staff. Auto Insurance claims industry experience preferred.
AMERICANS WITH DISABILITY SPECIFICATIONS
PHYSICAL DEMANDS
The physical demands described here are representative of those that must be met by an employee to successfully perform the essential functions of this job. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.
While performing the duties of this job, the employee is occasionally required to stand; walk; sit; use hands to finger, handle, or feel objects, tools, or controls; reach with hands and arms; climb stairs; balance; stoop, kneel, crouch or crawl; talk or hear; taste or smell. The employee must occasionally lift and/or move up to 25 pounds. Specific vision abilities required by the job include close vision, distance vision, color vision, peripheral vision, depth perception, and the ability to adjust focus.
WORK ENVIRONMENT
Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.
AF is a remote working environment.