Last updated on: 29th August, 2025
Hi folks! I had been meaning to write this for a while but kept holding up on it because I felt like I'd be blank if I tried. I realise now that that was just my brain playing tricks on me.
I've been working on Transformers for a while now! Close to a month. I made the jump (from statistical models to encoder-decoders - an almost 20 year jump!). And what's cooler is that I was able to implement the paper by myself. I understand that it's a silly thing but it left me feeling much more accomplished for some reason! I also just implemented Rotary Positional Encodings after trying to wrap my head around the paper for the past week. It is a very very cool technique and surprising that it even works.
Check out my work here: GPT and RoPE
I'm also going to participate in GovHack which happens every year, but always on a weekend, and I work most weekends 😭 That's why I didn't participate last year, but not letting that stop me this time. It's starting at 7PM today and ending at 5PM on Sunday - such a short time!!!!!
I can now play Experience by Ludovico Einaudi on the piano LESGOOOOOOO. Boredom is such a huge part of the process btw, you cannot git gud if you cannot stand being bored. I keep a few songs on the side that I play just for fun and don't try and be "good" at them. One of them is Supermarket Flowers by Ed Sheeran.
I recently finished reading Anti Fragile by Nassim Nicholas Taleb, and it might just be one of my favorites (haven't even read that many books but still). It really introduces a concept and takes it all the way, I mean, ALL THE WAY. I think that is very honorable. I'm going to read The Fifth Science by exurb1a who I didn't even know wrote books until now.
Anyway, I'm outta here.