Shrive Technologies
Overview
Gen AI Architect at Shrive Technologies
Location: Oakland, CA Salary: $75,000.00-$80,000.00 per year Join to apply for the Gen AI Architect role at Shrive Technologies. Note: This description preserves the original role focus and core requirements without extraneous postings. Responsibilities
Set up and deploy Langfuse v3 in the production environment. Architect and implement the upgrade of Langfuse v2 to v3 within the LamBots framework, ensuring backward compatibility and performance optimization. Design modular components for prompt management, tracing, metrics, evaluation, and playground features using Langfuse v3. Leverage Langfuse's full feature set: prompt management versioning, templating, and optimization; tracing end-to-end visibility into GenAI workflows; metrics performance, latency, and usage analytics; evaluation automated and manual scoring of model outputs; playground interactive testing and debugging of prompts. Integrate Azure AI Evaluation SDK into LamBots to enable scalable enterprise-grade evaluation pipelines/workflows. Build reusable components and templates for evaluation across diverse GenAI use cases. Collaborate with cross-functional teams to integrate evaluation capabilities into production pipelines/systems. Ensure scalability and reliability of evaluation tools in both offline and online environments. Define and enforce evaluation standards and best practices for GenAI agents, RAG pipelines, and multi-agent orchestration. Collaborate with product, engineering, and data science teams to align evaluation metrics with business KPIs. Drive observability, debugging, and traceability features for GenAI workflows. Stay current with emerging GenAI evaluation tools, frameworks, and methodologies. Qualifications
Hands-on experience with Langfuse (including v3 features) and integrations. Experience with other GenAI observability tools (e.g., TruLens, W&B, Helicone). Knowledge of Retrieval-Augmented Generation (RAG), fine-tuning, and multi-agent orchestration. Strong understanding of Azure AI services, especially the Evaluation SDK. Deep expertise in LLMOps, prompt engineering, and GenAI lifecycle management. Proficiency in Python, TypeScript, or similar languages used in GenAI frameworks. Experience with cloud-native architectures (Azure preferred). Familiarity with tracing tools, observability platforms, and evaluation metrics. Excellent communication and documentation skills.
#J-18808-Ljbffr
Location: Oakland, CA Salary: $75,000.00-$80,000.00 per year Join to apply for the Gen AI Architect role at Shrive Technologies. Note: This description preserves the original role focus and core requirements without extraneous postings. Responsibilities
Set up and deploy Langfuse v3 in the production environment. Architect and implement the upgrade of Langfuse v2 to v3 within the LamBots framework, ensuring backward compatibility and performance optimization. Design modular components for prompt management, tracing, metrics, evaluation, and playground features using Langfuse v3. Leverage Langfuse's full feature set: prompt management versioning, templating, and optimization; tracing end-to-end visibility into GenAI workflows; metrics performance, latency, and usage analytics; evaluation automated and manual scoring of model outputs; playground interactive testing and debugging of prompts. Integrate Azure AI Evaluation SDK into LamBots to enable scalable enterprise-grade evaluation pipelines/workflows. Build reusable components and templates for evaluation across diverse GenAI use cases. Collaborate with cross-functional teams to integrate evaluation capabilities into production pipelines/systems. Ensure scalability and reliability of evaluation tools in both offline and online environments. Define and enforce evaluation standards and best practices for GenAI agents, RAG pipelines, and multi-agent orchestration. Collaborate with product, engineering, and data science teams to align evaluation metrics with business KPIs. Drive observability, debugging, and traceability features for GenAI workflows. Stay current with emerging GenAI evaluation tools, frameworks, and methodologies. Qualifications
Hands-on experience with Langfuse (including v3 features) and integrations. Experience with other GenAI observability tools (e.g., TruLens, W&B, Helicone). Knowledge of Retrieval-Augmented Generation (RAG), fine-tuning, and multi-agent orchestration. Strong understanding of Azure AI services, especially the Evaluation SDK. Deep expertise in LLMOps, prompt engineering, and GenAI lifecycle management. Proficiency in Python, TypeScript, or similar languages used in GenAI frameworks. Experience with cloud-native architectures (Azure preferred). Familiarity with tracing tools, observability platforms, and evaluation metrics. Excellent communication and documentation skills.
#J-18808-Ljbffr