Embeddings
AI & ML: lesson 3 of 32
Meaning turned into a fixed list of numbers.
Lesson 3 of 32 · 5 min
Embeddings
Step 1 of 11
dims2 (real: hundreds+)
An embedding turns text into a fixed-length list of numbers — a position in space.
The Idea
An embedding maps text, images or audio to a fixed-length list of numbers. Models are trained so related items land near each other. Once meaning is geometry, searching and grouping become arithmetic.
Real-World Example
A paint shop's colour matcher reads the chip off your wall as a handful of numbers, then walks to the nearest tin on the rack. Close in numbers means indistinguishable to your eye.
The Code
vecs = {"cat": [0.9, 0.1], "kitten": [0.8, 0.2], "car": [0.1, 0.9]}
def embed(words): # crude sentence embedding
total = [0.0, 0.0]
for w in words:
for i, v in enumerate(vecs[w]):
total[i] += v
return [round(t / len(words), 2) for t in total]
print(embed(["cat", "kitten"])) # [0.85, 0.15] still in animal territory
print(embed(["cat", "car"])) # [0.5, 0.5] a meaningless midpointYour turn
Fill in the blank.
vecs = {"sun": [1.0, 0.0], "moon": [0.8, 0.2]}
total = [0.0, 0.0]
for w in ["sun", "moon"]:
for i, v in enumerate(vecs[w]):
total[i] += v
# average the two vectors -> want [0.9, 0.1]
print([round(t / ___, 2) for t in total])Mini quiz
1 / 3