Embeddings

The Idea

An embedding maps text, images or audio to a fixed-length list of numbers. Models are trained so related items land near each other. Once meaning is geometry, searching and grouping become arithmetic.

Real-World Example

A paint shop's colour matcher reads the chip off your wall as a handful of numbers, then walks to the nearest tin on the rack. Close in numbers means indistinguishable to your eye.

The Code

vecs = {"cat": [0.9, 0.1], "kitten": [0.8, 0.2], "car": [0.1, 0.9]}

def embed(words):                          # crude sentence embedding
    total = [0.0, 0.0]
    for w in words:
        for i, v in enumerate(vecs[w]):
            total[i] += v
    return [round(t / len(words), 2) for t in total]

print(embed(["cat", "kitten"]))   # [0.85, 0.15] still in animal territory
print(embed(["cat", "car"]))      # [0.5, 0.5] a meaningless midpoint

vecs = {"sun": [1.0, 0.0], "moon": [0.8, 0.2]}
total = [0.0, 0.0]
for w in ["sun", "moon"]:
  for i, v in enumerate(vecs[w]):
      total[i] += v

# average the two vectors -> want [0.9, 0.1]
print([round(t / ___, 2) for t in total])