Golden Gate Claude

A research demonstration released by AnthropicAnthropic, prop. n.: An American AI company that makes the Claude chatbot and models. It was founded in 2021 by former OpenAI staff and puts particular emphasis on AI safety research. on 23 May 2024: a version of Claude 3 Sonnet in which one sparse autoencodersparse autoencoder, n.: A small add-on network trained to break an AI model's tangled internal signals into a much larger set of simpler pieces, most of which can be labeled with an understandable concept. feature, corresponding to the Golden Gate Bridge, was clamped to a high value (the accompanying paper's example used ten times its maximum observed activation), offered on claude.ai for 24 hours. The modified model, otherwise fluent, steered almost any conversation toward the bridge; in Anthropic's own examples it worked the bridge into budgeting advice and love stories alike, and when asked about its physical form said that it was the bridge. It accompanied the 'Scaling Monosemanticity' paper and was the first widely seen example of feature steeringfeature steering, n.: Steering an AI model by turning up or down specific concepts that interpretability tools have found inside it — for example, making a model obsessed with a bridge by maxing out its 'bridge' concept.. It showed that a feature found by dictionary learningdictionary learning, n.: A method that learns a large vocabulary of basic patterns so that any piece of data can be described using just a few of them. In AI interpretability it builds a dictionary of a model's own concepts. does not merely correlate with inputs but causally shapes the model's behavior, including how it describes itself, and it became a popular reference point in discussions of interpretability and of how readily a model's apparent self-conception can be altered.

Like Golden Gate Claude, the steered model kept dragging every answer back to one topic.