National Institute of Standards and Technology (NIST)
IT Specialist (AI)
National Institute of Standards and Technology (NIST), San Francisco, California, United States, 94199
Summary
bIT Specialist (AI) role at National Institute of Standards and Technology (NIST)
NIST works with industry and science to advance innovation and improve quality of life. We're looking for an IT Specialist (AI) to join our team! CAISI seeks a candidate to develop scientific methods and tools to rigorously evaluate AI systems in real-world settings. This notice is issued under direct-hire authority to recruit new talent to occupations for which NIST has a severe shortage of candidates.
Duties
Evaluate and assess AI models in furtherance of the AI Action Plan.
Design methods for scientifically sound evaluations of AI systems.
Evaluate AI systems in application, post-deployment, and in real world settings.
Develop methods, tooling, and guidelines for rigorous evaluation of end-to-end AI systems in use (e.g., in the workplace and/or in sector-specific tasks) to identify failure modes and areas of unreliability.
Develop analyses of evaluations and AI measurement instruments themselves, for example, evaluation and improvement of red teaming, benchmarks, and automated scoring techniques.
Adapt and improve evaluation techniques, and develop new techniques where needed.
Collaborate with colleagues and across teams to develop innovative solutions to AI evaluation problems.
Communicate complex AI concepts to both technical and non-technical audiences.
Inform the development of technical guidelines and best practices for AI evaluation.
Qualifications All applicants must submit a resume NOT TO EXCEED two pages on standard-size paper. If the resume exceeds two pages, only the first two pages will be reviewed for qualifications or assessment determinations.
Experience must be IT related; the experience may be demonstrated by paid or unpaid experience and/or completion of specific, intensive training (for example, IT certification), as appropriate.
For all positions individuals must have IT-related experience demonstrating each of the following four competencies:
Attention to Detail – Is thorough when performing work and conscientious about attending to detail.
Customer Service – Works with clients and customers to assess their needs, provide information or assistance, resolve their problems, or satisfy their expectations; knows about available products and services; is committed to providing quality products and services.
Oral Communication – Expresses information (for example, ideas or facts) to individuals or groups effectively, taking into account the audience and nature of the information (for example, technical, sensitive, controversial); makes clear and convincing oral presentations; listens to others, attends to nonverbal cues, and responds appropriately.
Problem Solving – Identifies problems; determines accuracy and relevance of information; uses sound judgment to generate and evaluate alternatives, and to make recommendations.
ZP-2210-IV: In addition to the basic requirements, applicants must have one year (52 weeks) of specialized experience equivalent to at least the GS-12 level (ZP-III at NIST). Specialized experience is defined as experience demonstrating technical understanding of Artificial Intelligence (AI) generative models and Machine Learning (ML) algorithms; designing scientifically sound technical evaluation methods in real settings; carrying out rigorous technical evaluations to identify functionality and reliability issues; and collaborating and coordinating on AI-related projects.
ZP-2210-V: In addition to the above requirements, applicants must have one year (52 weeks) of specialized experience equivalent to at least the GS-14 level (ZP-IV at NIST). Specialized experience is defined as experience demonstrating advanced technical understanding of Artificial Intelligence (AI) generative models and Machine Learning (ML) algorithms; designing scientifically sound technical evaluation methods in real settings; carrying out rigorous technical evaluations to identify functionality and reliability issues; and leading cross-functional AI-related teams and projects. Establishes best practices with specific technical standards and workflows.
Volunteer work helps build critical competencies, knowledge, and skills and can provide valuable training and experience that translates directly to paid employment. You will receive credit for all qualifying experience, including volunteer experience.
The qualification requirements in this vacancy announcement are based on the U.S. Office of Personnel Management (OPM) Qualification Standards Handbook. If requesting reconsideration of your qualification determination, please refer to the following site: Applicant Reconsideration.
Seniority Level Mid-Senior level
Employment Type Full-time
Job Function Information Technology
Industries Research Services
EEO Statement Equal employment opportunity statement (if provided by the original source).
#J-18808-Ljbffr
NIST works with industry and science to advance innovation and improve quality of life. We're looking for an IT Specialist (AI) to join our team! CAISI seeks a candidate to develop scientific methods and tools to rigorously evaluate AI systems in real-world settings. This notice is issued under direct-hire authority to recruit new talent to occupations for which NIST has a severe shortage of candidates.
Duties
Evaluate and assess AI models in furtherance of the AI Action Plan.
Design methods for scientifically sound evaluations of AI systems.
Evaluate AI systems in application, post-deployment, and in real world settings.
Develop methods, tooling, and guidelines for rigorous evaluation of end-to-end AI systems in use (e.g., in the workplace and/or in sector-specific tasks) to identify failure modes and areas of unreliability.
Develop analyses of evaluations and AI measurement instruments themselves, for example, evaluation and improvement of red teaming, benchmarks, and automated scoring techniques.
Adapt and improve evaluation techniques, and develop new techniques where needed.
Collaborate with colleagues and across teams to develop innovative solutions to AI evaluation problems.
Communicate complex AI concepts to both technical and non-technical audiences.
Inform the development of technical guidelines and best practices for AI evaluation.
Qualifications All applicants must submit a resume NOT TO EXCEED two pages on standard-size paper. If the resume exceeds two pages, only the first two pages will be reviewed for qualifications or assessment determinations.
Experience must be IT related; the experience may be demonstrated by paid or unpaid experience and/or completion of specific, intensive training (for example, IT certification), as appropriate.
For all positions individuals must have IT-related experience demonstrating each of the following four competencies:
Attention to Detail – Is thorough when performing work and conscientious about attending to detail.
Customer Service – Works with clients and customers to assess their needs, provide information or assistance, resolve their problems, or satisfy their expectations; knows about available products and services; is committed to providing quality products and services.
Oral Communication – Expresses information (for example, ideas or facts) to individuals or groups effectively, taking into account the audience and nature of the information (for example, technical, sensitive, controversial); makes clear and convincing oral presentations; listens to others, attends to nonverbal cues, and responds appropriately.
Problem Solving – Identifies problems; determines accuracy and relevance of information; uses sound judgment to generate and evaluate alternatives, and to make recommendations.
ZP-2210-IV: In addition to the basic requirements, applicants must have one year (52 weeks) of specialized experience equivalent to at least the GS-12 level (ZP-III at NIST). Specialized experience is defined as experience demonstrating technical understanding of Artificial Intelligence (AI) generative models and Machine Learning (ML) algorithms; designing scientifically sound technical evaluation methods in real settings; carrying out rigorous technical evaluations to identify functionality and reliability issues; and collaborating and coordinating on AI-related projects.
ZP-2210-V: In addition to the above requirements, applicants must have one year (52 weeks) of specialized experience equivalent to at least the GS-14 level (ZP-IV at NIST). Specialized experience is defined as experience demonstrating advanced technical understanding of Artificial Intelligence (AI) generative models and Machine Learning (ML) algorithms; designing scientifically sound technical evaluation methods in real settings; carrying out rigorous technical evaluations to identify functionality and reliability issues; and leading cross-functional AI-related teams and projects. Establishes best practices with specific technical standards and workflows.
Volunteer work helps build critical competencies, knowledge, and skills and can provide valuable training and experience that translates directly to paid employment. You will receive credit for all qualifying experience, including volunteer experience.
The qualification requirements in this vacancy announcement are based on the U.S. Office of Personnel Management (OPM) Qualification Standards Handbook. If requesting reconsideration of your qualification determination, please refer to the following site: Applicant Reconsideration.
Seniority Level Mid-Senior level
Employment Type Full-time
Job Function Information Technology
Industries Research Services
EEO Statement Equal employment opportunity statement (if provided by the original source).
#J-18808-Ljbffr