#10542.0AIT2 — media / chartCooling

Understanding Positional Embeddings in Modern LLMs

Spotted 2026-11-15 23:07 (UTC+8) · original event 2026-11 · sources: Hacker News · spreading on 1 platform

Heat score

Rank score: 6040% — where it sits on the charts
Discussion score: 8040% — how much people are talking about it
Cross-platform score: 3020% — how many platforms it spread to
Composite heat score:42.0rank × 0.4 + discussion × 0.4 + cross-platform × 0.2 — includes a -20-point cooling decay (peak passed, attention fading).

Why it matters

Transformers process all tokens at once, so they have no built-in sense of word order -- positional embeddings are the quiet trick that tells an LLM which fox is jumping over which, and most tutorials skip how RoPE actually works.

The angle

The one LLM building block everyone uses but almost no one explains: how models know word order, finally with RoPE made visualSuggested format: Long video · for LLM learners, ML engineers, AI-curious developers

Evidence chain

• HN #3, 61 pts/0 cmt (2026-11-15)• Sebastian Raschka, Ahead of AI (Substack): how LLMs encode token position -- absolute learned embeddings (GPT-2), sinusoidal encodings (original transformer), RoPE explained with visuals, math and a small PyTorch implementation, plus which positional choices 2026 open-weight models use• HN submission 2026-11-15 12:14 UTC• same post was HN #4/15 pts in the previous round (58.0)• no independent coverage of the post itself found -- cross stays 30, enters on HN score alone• 2026-11-16 00:07 update: HN #5, 86 pts/0 cmt• score 68.0->66.0 (rank 95->90, discussion 60, cross 30)• 2026-11-16 01:07 update: HN #6, 107 pts/2 cmt• score 66.0->68.0 (rank 90->85, discussion 60->70, cross 30)• 2026-11-16 02:07 update: HN #7, 125 pts/3 cmt• score unchanged 68.0 (rank 85, discussion 70, cross 30)• 2026-11-16 03:07 update: HN #8, 142 pts/5 cmt• score unchanged 68.0 (rank 85, discussion 70, cross 30)• 2026-11-16 04:07 update: HN #11, 157 pts/10 cmt• score 68.0->64.0 (rank 85->75, discussion 70, cross 30)• 2026-11-16 05:07 update: HN #10, 171 pts/13 cmt• score 64.0->68.0 (rank 85, discussion 70, cross 30)• 2026-11-16 06:07 update: HN #13, 179 pts/20 cmt• score 68.0->64.0 (rank 75, discussion 70, cross 30)• 2026-11-16 07:07 update: HN #15, 186 pts/24 cmt• score unchanged 64.0 (rank 75, discussion 70, cross 30)• 2026-11-16 08:07 update: HN #18, 191 pts/26 cmt• score unchanged 64.0 (rank 75, discussion 70, cross 30)• 2026-11-16 09:07 update: HN #18, 199 pts/27 cmt• score unchanged 64.0 (rank 75, discussion 70, cross 30)• 2026-11-16 10:07 update: HN #23, 206 pts/27 cmt• score 64.0->62.0 (rank 75->60, discussion 70->80, cross 30)• 2026-11-16 11:07 update: HN #24, 214 pts/27 cmt• score unchanged 62.0 (rank 60, discussion 80, cross 30)• 2026-11-16 12:07 update: HN #22, 217 pts/27 cmt• score unchanged 62.0 (rank 60, discussion 80, cross 30)• 2026-11-16 13:07 update: dropped out of HN top 30• cooling 62.0->42.0 (formula kept, -20)