AI by Hand ✍️ SOTA Packet - April Delivery

The cutoff date has passed for the April delivery. Email me tom@byhand.ai if you wish to join the waitlist.

Cutoff date: 3/21/25 (Friday) 

Shipping date: 4/1/25 (Tuesday) 

The March delivery of the first-ever SOTA packets are very well received! Thank you! 

What is a SOTA packet?

I will fill a large envelope with original study materials I designed for SOTA AI algorithms too new to include in my courses at my university and too advanced for beginners.

Each packet will be prepared by hand ✍️ by me. Each exercise I put into the packet will require you to work by hand in the same way I did when I studied these algorithms.

Why?

Many people told me they feel overwhelmed by all the new AI papers coming out all the time. I feel the same way. I want to help you cut through the noise and study what's really important.

How?

I spent a lot of time to study and design these exercises in the AI by Hand style. Instead of reading pages of text and math equations, you see all the calculations in visual blocks of matrix multiplications with concrete numbers. You can use the least amount of time to grasp the most important mathematical principles behind each algorithm or architecture.

Table of Content

1. Attention:
   1. Multi-Head Attention (MHA);
   2. Grouped Query Attention (GQA);
   3. Multi-Head Latent Attention (MLA);
   4. Native Sparse Attention (MSA).
2. Feed-Forward Network:
   1. Positional-Wise Feed-Forward Network;
   2. Mixture of Experts;
   3. Sparse Mixture of Experts.
3. GPU:
   1. Tiled Matrix Multiplication;
   2. Systolic Array;
   3. Flash Attention
4. Techniques:
   1. BatchNorm;
   2. LayerNorm;
   3. RMSNorm;
   4. Low-Rank Adaptation (LoRA);
   5. Rotary Positional Embedding (RoPE)
5. Alternatives to Transformers:
   1. Kolmogorov-Arnold Networks (KAN);
   2. LSTM;
   3. xLSTM;
   4. Receptance Weighted Key Value (RWKV).