What Is an LLM

The Idea

A large language model has one job: given the tokens so far, score every possible next token. Answering, translating and writing code all fall out of running that prediction in a loop, feeding each choice back in.

Real-World Example

A radio announcer who has read the shipping forecast for twenty years can finish a sentence before it is spoken. The phrasing is that patterned — no understanding of the weather required.

The Code

text = "the cat sat on the mat the cat ran"
words = text.split()

nxt = {}
for a, b in zip(words, words[1:]):      # count what follows what
    nxt.setdefault(a, []).append(b)

def most_likely(word):                  # a real model conditions on
    options = nxt[word]                 # thousands of tokens, not one
    return max(set(options), key=options.count)

print(nxt["the"])            # ['cat', 'mat', 'cat']
print(most_likely("the"))    # cat

text = "red fish blue fish red fish red"
words = text.split()

nxt = {}
for a, b in zip(words, words[1:]):
  nxt.setdefault(a, []).append(b)

opts = nxt["fish"]
print(max(set(opts), key=opts.count))