AI & Model Training

This page explains the AI stack end to end: what runs on every message, which models are self-hosted vs. called as APIs, and how custom fine-tuning works.

The processing pipeline

Every message runs through the same ordered steps, all inside the rezolyn-platform service:

1. Language detection — a classifier (XLM-RoBERTa) identifies the language; a lightweight detector backs it up on short/code-switched text.
2. Translation to English — MarianMT models translate to English for the next step. This is an internal optimisation, not the product — if it fails, the pipeline still answers.
3. Intent classification — the message is sorted into one of eight intents (balance, loan status, transaction history, account access, complaint, product query, handoff, or open).
4. Response — a known intent with high confidence uses a deterministic template (fast, safe, no LLM). Everything else goes to an LLM under a strict guardrail prompt. If the LLM is unavailable, the pipeline degrades to a safe "an agent will follow up" template — it never fails the customer.

Where the models run

There are two kinds, and the distinction matters for cost, latency, and where training happens.

Self-hosted (open models, run on our GPUs)

Language detection, MarianMT translation, the intent classifier, and fine-tuned Whisper (for the moat languages: Yoruba, Hausa, Igbo, Twi, Wolof) run inside the platform on a GPU worker. No per-call API cost, full control, and the place custom training lands.

Managed (called as APIs)

• LLM responses route through a flexible multi-provider layer — Claude, GPT, Groq (Llama), Mistral, xAI Grok, or Gemini. Each model is an environment variable, so you can switch provider without a code change, and providers are tried in order so an outage degrades to the next one.
• Speech-to-text for non-moat languages uses AssemblyAI (voice notes) or Deepgram (low-latency IVR), with self-hosted Whisper for the moat languages.
• Text-to-speech uses Google Cloud TTS, with ElevenLabs for premium voices.

Custom model training

You improve the models on your own conversations. This is real, and it works in three stages.

1. Curate training data

As conversations come in, flag the good ones into your training set:

POST /v1/conversations/{session_id}/training

Export the curated set any time as JSONL — the exact format the trainer consumes:

GET /v1/conversations/export?project_id=…&training_only=true

Each line is one labelled example: { "text", "translation", "intent" }.

2. Start a fine-tune

On the Scale plan (with fine-tuning enabled), start a job:

curl -X POST https://api.rezolyn.com/v1/training/jobs \
  -H "Authorization: Bearer <session-jwt>" \
  -d '{ "language": "yo", "model_type": "intent" }'

The API gates this: you must have enough curated examples (50+ per language), be on a plan that includes fine-tuning, and have an owner/admin role. It then creates a tracked job and dispatches it to a GPU worker.

3. Track it

GET /v1/training/jobs          # all your jobs
GET /v1/training/jobs/{id}     # status + progress %

Jobs move queued → running → succeeded / failed, with live progress. A finished intent model is published and picked up on the next config roll.

Knowledge base (RAG)

Upload your FAQs and docs; Rezolyn chunks and embeds them into a pgvector store scoped to your project. On product and open queries, the most relevant chunks are retrieved into the LLM's context, so answers are grounded in your content. Enabled per-project once embeddings are configured.

Guardrails

The LLM runs under a strict system prompt it cannot override: never invent financial figures or account data, never generate credentials or OTPs, never try to resolve complaints autonomously, and route to a human whenever it's unsure. Outputs are validated before they're sent.