Embeddings — Explained Simply

What Embeddings Actually Are

You Are Moving to a New City

You have just moved to Lagos. You need to describe yourself to new people so they can understand who you are and connect you with others like you.

Words work for humans but computers cannot measure similarity from words alone. To answer "who in this city is most similar to me?" you need to convert your description into numbers — a list that captures who you are in a way that can be measured and compared.

Your Description as Numbers

Samuel's embedding:
  Technical skill:    0.92
  Creativity:         0.74
  Ambition:           0.88
  Social energy:      0.61
  Learning drive:     0.95

As a vector: [0.92, 0.74, 0.88, 0.61, 0.95]

Now you are a point in space. Other people are also points in the same space:

Person A: [0.91, 0.70, 0.85, 0.58, 0.93]  ← very close to you
Person B: [0.20, 0.95, 0.30, 0.90, 0.25]  ← very different

Distance in this space = similarity in real life.

That list of numbers — representing a person, word, sentence, or document — is an embedding.

Why Words Need Embeddings

Assigning raw integers to words is useless:

"Lagos"   → 2
"Nigeria" → 4

The number gap of 2 suggests they are very different.
But they are closely related places.

Embeddings fix this by putting related words close together in space.

Word Embeddings — Words as Points in Space

Lagos    → [0.82, 0.45, 0.91, 0.12, 0.67]
Nigeria  → [0.80, 0.43, 0.89, 0.11, 0.65]  ← very close — related places
London   → [0.78, 0.51, 0.84, 0.09, 0.71]  ← similar (city) but different
Cat      → [0.12, 0.88, 0.03, 0.95, 0.21]  ← far away — different concept

Arithmetic in Embedding Space

King - Man + Woman ≈ Queen

embedding("King") - embedding("Man") + embedding("Woman")
≈ embedding("Queen")

The model discovered that gender and royalty are independent
dimensions in meaning-space — without being explicitly told.

The Embedding Layer in PyTorch

import torch.nn as nn

embedding = nn.Embedding(
    num_embeddings = 10000,    # vocabulary size
    embedding_dim  = 256       # each word = 256 numbers
)

word_indices = torch.tensor([42, 156, 3, 891])
embedded     = embedding(word_indices)    # Shape: (4, 256)

print(embedded.shape)   # torch.Size([4, 256])
# Each of the 4 words is now a 256-number vector

Sentence Embeddings — Whole Sentences as Points

"This product is absolutely amazing"
→ [0.92, 0.15, 0.88, ...]   ← positive sentiment region

"I am so disappointed with this"
→ [-0.81, 0.89, -0.72, ...]  ← negative sentiment region

"The packaging was okay"
→ [0.12, 0.45, 0.08, ...]    ← neutral region

Sentences with similar sentiment end up in similar regions. This is how your sentiment classifier works — it maps Nigerian Pidgin reviews into embedding space, then reads where they landed.

The Full Pipeline — How Embeddings Connect to Transformers

Input: "Dis product dey good"

Step 1 — Tokenise:    ["Dis", "product", "dey", "good"]
Step 2 — Embed:       [ [0.82,...], [0.45,...], [0.91,...], [0.12,...] ]
Step 3 — Transform:   Transformer layers mix and rotate these vectors
Step 4 — Output:      Final vector → sentiment prediction

The entire intelligence of a language model lives in
how it moves things around in embedding space.

Embeddings in Your Nigerian Pidgin Research

Before fine-tuning:
  "e too good" → embedding in general language space
  Model does not know this means positive in Pidgin

After fine-tuning on your labelled data:
  "e too good" → embedding shifts toward positive region
  "e no good"  → embedding shifts toward negative region
  "e so so"    → embedding shifts toward neutral region

Your labelled dataset teaches the model where Nigerian Pidgin sentiments live in embedding space.

The Real Words Mapped to the Story

The One Thing to Remember

An embedding turns something computers cannot understand — a word, sentence, or person — into a list of numbers that captures its meaning. Similar things end up close together in this number-space. Everything in modern NLP — BERT, GPT, LLaMA, AfroXLMR — starts by converting text to embeddings.