AI Operations Specialist – LLM & Conversation Systems
Position Type: Full-Time, Remote
Working Hours: U.S. Business Hours
About the Role
We are looking for a hands-on AI Operations Specialist to own and improve our production AI conversation systems.
We are an AI-powered SMS lead recovery platform serving telehealth brands, processing millions of messages each month. Our AI systems interact directly with customers, and their performance directly impacts client conversions.
In this role, you'll onboard new clients, configure AI conversation systems, monitor live conversations, diagnose issues, improve prompts and system behavior, and measure whether your changes actually improve conversion.
This is not a theoretical AI role. We're looking for someone who has run LLM systems in production and understands how to continuously monitor, troubleshoot, test, and improve them.
ResponsibilitiesClient Onboarding & AI Setup
Onboard new clients into the AI conversation platform.
Set up webhook integrations and client knowledge bases.
Configure campaigns and conversation workflows.
Ensure new client systems are properly tested before going live.
Troubleshoot integration and configuration issues during onboarding.
Production AI Operations
Monitor live AI-generated conversations and customer interactions.
Review conversations to identify errors, inconsistencies, and missed opportunities.
Diagnose AI failure modes and determine their root causes.
Identify issues affecting conversation quality and conversion.
Escalate technical issues when necessary and follow through until resolved.
Prompt & System Optimization
Write, edit, and improve prompts that generate customer-facing messages.
Adjust system instructions and conversation logic based on real-world performance.
Test changes systematically before and after deployment.
Maintain documentation of prompt and system changes.
Continuously improve AI behavior while maintaining accuracy and consistency.
Performance & Conversion Analysis
A
📌 AI Operations Specialist (Perú)
🏢 Pavago
📍 Perú