Job Title

Junior Applied AI Engineer (LLM Systems)

Key Skills

LLMs, Generative AI, AWS Bedrock, Prompt Engineering, AI Agent Evaluation, Langfuse, Python, FastAPI, MCP, Guardrails & Moderation, REST APIs, Git

Experience

1-3+ years of experience in AI/ML Engineering, Applied AI, Data Science, or related fields.

Location

LATAM (Remote)

Mode

Remote

We at Coforge are hiring Junior Applied AI Engineer (LLM Systems) (20848-10-1) with the following skill set.

Key Responsibilities

- Design and execute evaluation frameworks for AI agents, including test datasets, LLM-as-a-Judge methodologies, human reviews, and regression testing.
- Build, configure, and optimize AI guardrails and moderation systems for regulated domains such as financial services and credit products.
- Develop, test, and continuously improve prompts based on evaluation results and production trace analysis.
- Create and maintain MCP (Model Context Protocol) servers and tool integrations that enable AI agents to interact with internal data and services.
- Support fine-tuning and evaluation of language models using domain-specific datasets.
- Prototype and validate AI workflows using System One models such as Jev and Laya for classification, routing, and verification tasks.
- Monitor AI system quality, latency, performance, and costs through Langfuse and implement improvement initiatives.
- Collaborate with Engineering, Data Science, and Compliance teams to ensure AI solutions are reliable, secure, and compliant.

Required Skills & Qualifications

- 1-3+ years of experience in Machine Learning, Applied AI, Data Science, or Software Engineering.
- Experience working with Large Language Models (LLMs)



and AI-powered applications.
- Knowledge of LLM evaluation techniques, including benchmark datasets, scoring frameworks, and LLM-as-a-Judge approaches.
- Understanding of AI agent behavior across multi-turn conversations and tool-calling workflows.
- Experience with prompt engineering, structured outputs, and prompt optimization.
- Familiarity with AI safety concepts including moderation, prompt injection prevention, jailbreak mitigation, and guardrails.
- Hands-on experience with AWS cloud services, preferably Amazon Bedrock.
- Experience with Langfuse or similar observability and evaluation platforms for LLM applications.
- Strong programming skills in Python.
- Experience developing REST APIs using FastAPI or similar frameworks.
- Knowledge of software engineering best practices including unit testing, code reviews, and Git-based development workflows.
- Advanced English communication skills (written and verbal).

Preferred Skills

- Experience developing MCP (Model Context Protocol) servers.
- Hands-on experience with fine-tuning LLMs using techniques such as LoRA or full fine-tuning.
- Familiarity with Small Language Models (SLMs), model distillation, and deployment strategies.
- Exposure to System One models such as Jev and Laya.
- Experience in financial services, fintech, banking, or other regulated industries.
- Knowledge of AI evaluation, observability, and production monitoring best practices.

Posted On

27-10-2026

At Coforge, we hire professionals based solely on their skills and qualifications and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.

📌 Junior Applied AI Engineer (LLM Systems) (Lima)
🏢 Encora
📍 Lima