e8_slp1_bpe8k
d20m (23.2M parameters) on E8 screening slice, trained on RTX 3060 12GB (Home GPU). Started 11 Sep 2026, ended 11 Sep 2026.
Hypothesis
Writing Sanskrit in SLP1 transliteration instead of Devanagari changes model quality.
What we learned
Pooled 0.7619, the same as Devanagari BPE: the script is a wash for the model, so SLP1 was kept for its cleaner tokens.
Scores
Bits per byte, lower is better.
ex-Gītā (headline)
0.7707
bits per byte
ex-Gītā, clean_v1
—
bits per byte
Pooled, all five sets
0.7619
bits per byte
Validation split
0.6927
bits per byte
| Test set | E8-slp1-bpe |
|---|---|
| DCS gold (classical) | 0.7355 |
| Bhagavad-gītā (memorisation) | 0.5313 |
| Out of domain | 0.7713 |
| Prose | 0.7417 |
| Vedic (Ṛgveda) | 1.2938 |
Curves
Training loss
Cross-entropy per token, by step. The first few percent of the run, far higher, run off the top; hover or the table has every value.
Held-out bits per byte
The validation split, evaluated during training, by step. Lower is better.
Throughput
Tokens per second, by step.
Model
- Preset
- d20m
- Parameters
- 23,239,168
- Outside embeddings
- 18,881,024
- Layers · heads · width
- 6 · 8 · 512
- Tokenizer
- SLP1 BPE 8k
Data
- Slice
- E8 screening slice
- Words
- —
- Training tokens
- 592,953,023
- Passes
- 0.59 passes
- Tokens seen
- 348,143,616
Compute
- GPU
- RTX 3060 12GB
- Where
- Home GPU
- Steps
- 7,083 / 7,083
- GPU hours
- 0.78
- Cost
- —
- Spot restarts
- —
Lineage
homed20m