AI / NLP
GloVe from Scratch
Word Embeddings Trained on WikiText-2
About this project
GloVe word embeddings built from first principles: sparse co-occurrence matrix (120x smaller than dense), hand-derived gradients verified against autograd, and manual AdaGrad, with no autograd or torch.optim. Trained on WikiText-2 in 26 s on a 4 GB laptop GPU, with analogy tests, ablations and a comparison against official GloVe vectors.
- Category
- AI / NLP
- Built by
- Jeyadev R
- Source code
- github.com/jeyadev-jd/glove-wikitext2
