Veracity Software Inc
Data scientists, AI engineering, LLM engineer, Machine Learning Engineers
Veracity Software Inc, Charlotte, North Carolina, United States, 28245
Overview
Role: Data scientists, AI engineering, LLM engineer, Machine Learning Engineers Location: Charlotte, NC (Local candidate only) Video Interview Project Details Implemented a chatbot internally with the bank, build interface and now users can interact. It's a RAG framework so instead of tuning it into the actual applications you can prompt it to give you prevectorized queries. Able to feed it documents even if they don't know what your team is or who you are. Should have 1000 users by the end of the year and another 2000 next year.
Must Haves / Required Skills
LLMs & Inference: Experience with major LLMs, specifically Llama 3, Mistral, and possibly Quinn. Direct experience with VLLM (inference engine) is a strong match for handling batched requests. Experience with Nvidia Triton is a bonus and a key part of model serving infrastructure. Core Development: Python (mandatory); Python 3.12 in use, with 3.10+ acceptable. Web Frameworks: Flask or FastAPI (required for hosting the LLM via a Python endpoint). Java: Secondary preferred for creating REST services interacting with the front-end UI. Database & Data Management: Vector Databases (Redis and other vector stores) and SQL required. RAG Skills: Ability to interpret business-side parameters from the product team and push back if technically unfeasible; demonstrates critical thinking beyond coding. Infrastructure & Operations (MLOps) Containers & Orchestration: Knowledge of containers and OpenShift for CI/CD. CI/CD Tools: Experience with XLR and Datical for pipeline deployments. Hardware: Solid understanding of GPUs as a critical infrastructure component. Agile: Team uses Agile methodology. Scaling: Project growth tied to hardware availability; initial deployment capped at 1,000 users with scaling contingent on budget.
Skills / Experience That Are A Plus
Nice to have, but not necessarily required: General awareness of Vector DB vs relational databases. Experience pushing code to controlled environments and production AI applications. Any model experience or quantitative modeling, or prior white papers (as observed at the client).
Seniority level
Mid-Senior level
Employment type
Full-time
Job function
Other
Industries
IT Services and IT Consulting
#J-18808-Ljbffr
Role: Data scientists, AI engineering, LLM engineer, Machine Learning Engineers Location: Charlotte, NC (Local candidate only) Video Interview Project Details Implemented a chatbot internally with the bank, build interface and now users can interact. It's a RAG framework so instead of tuning it into the actual applications you can prompt it to give you prevectorized queries. Able to feed it documents even if they don't know what your team is or who you are. Should have 1000 users by the end of the year and another 2000 next year.
Must Haves / Required Skills
LLMs & Inference: Experience with major LLMs, specifically Llama 3, Mistral, and possibly Quinn. Direct experience with VLLM (inference engine) is a strong match for handling batched requests. Experience with Nvidia Triton is a bonus and a key part of model serving infrastructure. Core Development: Python (mandatory); Python 3.12 in use, with 3.10+ acceptable. Web Frameworks: Flask or FastAPI (required for hosting the LLM via a Python endpoint). Java: Secondary preferred for creating REST services interacting with the front-end UI. Database & Data Management: Vector Databases (Redis and other vector stores) and SQL required. RAG Skills: Ability to interpret business-side parameters from the product team and push back if technically unfeasible; demonstrates critical thinking beyond coding. Infrastructure & Operations (MLOps) Containers & Orchestration: Knowledge of containers and OpenShift for CI/CD. CI/CD Tools: Experience with XLR and Datical for pipeline deployments. Hardware: Solid understanding of GPUs as a critical infrastructure component. Agile: Team uses Agile methodology. Scaling: Project growth tied to hardware availability; initial deployment capped at 1,000 users with scaling contingent on budget.
Skills / Experience That Are A Plus
Nice to have, but not necessarily required: General awareness of Vector DB vs relational databases. Experience pushing code to controlled environments and production AI applications. Any model experience or quantitative modeling, or prior white papers (as observed at the client).
Seniority level
Mid-Senior level
Employment type
Full-time
Job function
Other
Industries
IT Services and IT Consulting
#J-18808-Ljbffr