smoke_d20m
d20m (23.2M parameters) on early test slice, trained on RTX 3060 12GB (Home GPU). Started 11 Sep 2026, ended 11 Sep 2026.
Hypothesis
First end-to-end check of the trainer on an early corpus slice.
What we learned
397M tokens in an hour on the home GPU. Superseded by E8.
Scores
Bits per byte, lower is better.
ex-Gītā (headline)
—
bits per byte
ex-Gītā, clean_v1
—
bits per byte
Pooled, all five sets
—
bits per byte
Validation split
0.6626
bits per byte
| Test set | Smoke test |
|---|---|
| DCS gold (classical) | — |
| Bhagavad-gītā (memorisation) | — |
| Out of domain | — |
| Prose | — |
| Vedic (Ṛgveda) | — |
Curves
Training loss
Cross-entropy per token, by step. The first few percent of the run, far higher, run off the top; hover or the table has every value.
Held-out bits per byte
The validation split, evaluated during training, by step. Lower is better.
Throughput
Tokens per second, by step.
Model
- Preset
- d20m
- Parameters
- 23,239,168
- Outside embeddings
- 18,881,024
- Layers · heads · width
- 6 · 8 · 512
- Tokenizer
- Devanagari unigram 8k
Data
- Slice
- early test slice
- Words
- —
- Training tokens
- —
- Passes
- —
- Tokens seen
- 396,853,248
Compute
- GPU
- RTX 3060 12GB
- Where
- Home GPU
- Steps
- 8,074 / 8,700
- GPU hours
- 1
- Cost
- —
- Spot restarts
- —
homed20m