cs & mathematics major @ ut austin interested in ml
contact me at femi [dot] bello [at] utexas [dot] edu
tl;dr: I built a simple version of PyTorch from scratch in C++. It supports most standard tensor operations, automatic differentiation, and can run a 135 Million paramter LLM on my M2 MacBook Air. The code is open source and available on GitHub. I go into the details of how I built it, why I made certain design choices, and what I learned from the process.
I recently spent a couple months working on a research project. The goal was to submit to NeurIPS’ new High School track. I’d never done something like this, and I learned a lot about doing ML research by trying.
I recently spent the better part of last week doing exactly what the title says.
A while ago I replicated the algorithm described by the paper Overcoming catastrophic forgetting in neural networks. I implemented everything in PyTorch in a colab notebook.