Latest

• Speculative Decoding for Fun and Profit III — 2026-08-18

  Recap: We are exploring the idea of leveraging a speculative decoding setup for extra work. In speculative decoding, we speed up decoding from a target model by hitching a draft model to it. The way this works is: we sample a bunch of tokens from the draft model, and validate them using the target model. It has been shown that this allows us to sample tokens more cheaply compared to sampling from the target model directly.

• Speculative Decoding for Fun and Profit II — 2026-06-25

  In Part I of this series of posts, we started exploring if the draft model in a speculative decoding setup can be leveraged for downstream tasks. Specifically, we proposed we could do this by appending a suffix prompt, only to the prompt for the draft model. Consider the specific case where the main task is document summarization, and the downstream task is to generate hashtags from that summary. Then the target model would receive the usual “summarize this document” prompt while the draft model would receive the prompt “summarize this document. then produce some hashtags”.

• Speculative Decoding for Fun and Profit I — 2026-06-17

  Take a prompt summarizing a document:

• LLMs and Napkin Problems — 2026-06-02

  On May 20th, Tim Gowers advised fellow mathematicians to sit down before reading the tweet that was to follow. In it, he declared that AI had solved Erdős’s unit-distance problem, a celebrated problem in discrete geometry first posed by Paul Erdős.

• Auto-Researching Tagore's Songs — 2026-05-18

  Rabindranath Tagore wrote roughly 2,000 songs — collectively Rabindra Sangeet. Earlier this year I curated them from widely available public sources into a structured dataset and put it on Hugging Face:

• The Chimeras of Tokenization — 2026-04-15

  The Byte Pair Encoding (BPE) method is odd!

• Clustering Graphs: Spectral vs HAC — 2026-04-02

  

  You are looking at a plot comparing two clustering algorithms on parameterized graph model. As the parameter changes, the graph transitions from one regime to another, the preferred algorithm changes (as evidenced by the y-axis: higher is better). The rest of the post will build up to this plot.

• Flash Attention in a Jiffy — 2026-03-18

  To celebrate the release of Flash Attention 4, I think it will be fun to work through the basic idea of Flash Attention. To keep things simple, we will focus on the forward pass and only on memory write traffic.

• AI v. Tagore — 2026-02-26

  Sarvam AI has garnered well-deserved attention for their recent sequence of “drops” of ML models for Indic languages (Bangla included). I’ve used and recommended their amazing dubbing model myself.

All posts →