shoggoth

A metaphor for a large language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. as an incomprehensible entity whose helpful assistant persona is only a thin surface. The canonical meme shows a many-eyed tentacled mass labeled as the pretrained model, a smaller face labeled supervised fine-tuningsupervised fine-tuning, n.: Teaching a pretrained AI model by showing it many examples of good answers to requests, so it learns to imitate them — like learning a job by studying worked examples., and a tiny smiley-face mask labeled RLHFreinforcement learning from human feedback, n.: A way of training AI using people's judgments: humans compare pairs of answers, a scoring model learns their preferences, and the AI is then trained to produce answers that score well.: the friendly chatbot is a mask that post-trainingpost-training, n.: Everything done to an AI model after its initial bulk training on huge amounts of text: the extra teaching that turns a raw text predictor into a helpful, safe assistant or a careful reasoner. fits over a model of internet text whose internal workings and dispositions remain poorly understood. The image condenses a worry central to interpretabilityinterpretability, n.: The effort to understand how an AI system arrives at its outputs — what information it uses and how it combines it — instead of treating it as a sealed black box. and AI alignmentAI alignment, n.: Making an AI system genuinely try to do what the people it serves want — not a literal-minded, distorted or self-serving version of it. Also the research field working on this. — that well-behaved outputs may not reveal what the system will do in unfamiliar situations — and is widely used as shorthand for the strangeness of large models.

One weird prompt and the smiley mask slips; you are talking to the shoggoth.