Mentor's 500M base, fully fine-tuned on 15,000 grounded legal/financial question-answer pairs. It turns the base completer into a model that answers a question from the provided context.
The 500M base given supervised fine-tuning on 15,000 grounded QA pairs — a full fine-tune of all 517.8M parameters, not an adapter. It learns to answer the question from the accompanying context rather than merely continue text.
2 epochs (2,814 steps). Scored on the same 500 held-out questions and the same blind 0–10 rubric judge the arena uses: 5.32/10 (groundedness 69.2%, fabrication 2.0%, token-F1 0.354). This is the checkpoint the 500M DPO and RLAIF models are aligned from.
| Class | LlamaForCausalLM |
| Layers | 24 |
| Hidden size | 1,280 |
| Attention | 20 heads · dim 64 · full MHA |
| Feed-forward | SwiGLU · inner 3,456 |
| Positional | RoPE · θ 10,000 |
| Norm | RMSNorm · ε 1e-5 |
| Context | 1,024 tokens |
| Vocabulary | 32,768 |
| Embeddings | tied input/output |
| Pretrained by | Dr. Sreedath Panat |
| Init from | slm-500m base |
| Method | Supervised fine-tune (full) |
| Trainable params | 517.8M — all |
| Training data | 15,000 QA pairs · 2 epochs |
| Optimizer steps | 2,814 |
our cost begins at fine-tuning — the base is imported.
| Stage | Detail | Cost |
|---|---|---|
| 500M fine-tuning | SFT + RAFT trained as one Modal app ($2.67 for both) — not separable per model | $2.67 |
| Total | $2.67 |
Figures are Modal GPU usage (time × rate) across this model's lineage; shared datasets are charged at this model's share. Whether base pretraining is included is stated above — it is for the models pretrained here, and excluded for imported bases. Evaluation-derived metrics come from an independent blind-judge harness on a frozen, decontaminated held-out set. Serving is billed separately and scales to zero.