Goodfire

A San Francisco start-up whose products and research aim to let developers see which concepts a neural network represents internally and change those representations directly. Its Ember platform (2024) offered hosted access to sparse autoencodersparse autoencoder, n.: A small add-on network trained to break an AI model's tangled internal signals into a much larger set of simpler pieces, most of which can be labeled with an understandable concept. features of open-weight language models, letting developers inspect them and steer behavior by amplifying or suppressing featuresfeature steering, n.: Steering an AI model by turning up or down specific concepts that interpretability tools have found inside it — for example, making a model obsessed with a bridge by maxing out its 'bridge' concept.; it has also released open sparse autoencoders for large models and applied mechanistic interpretabilitymechanistic interpretability, n.: A branch of AI research that tries to reverse-engineer a neural network — working out, piece by piece, which internal parts represent which concepts and how they combine to produce behavior. to scientific models, for example with the Arc Institute on the Evo 2 genomic model. It is among the best-known of a small group of startups seeking to commercialize interpretability.